Rodrigo Sanches Miani

CR
h-index12
3papers
1citation
Novelty22%
AI Score29

3 Papers

7.8CRMay 6
Evaluating the Reliability of Multiple Large Language Models in Risk Assessment: A CIS Controls Based Approach

Gustavo Roberto Pinto, Arthur do Prado Labaki, Rodrigo Sanches Miani

Proper implementation of technical and administrative controls reinforces an organization's cybersecurity posture and business resilience, reduces risks, and enhances governance, ultimately elevating business maturity. The dynamics of the technological landscape and emerging threats negatively affect the most diverse companies, regardless of their size. This, associated with a global gap in the cybersecurity workforce, imposes enormous challenges and the need for a profound change in how companies respond to threats. Generative Artificial Intelligence from large language models has become an influential tool across various companies, emerging as a viable option to help address those challenges while partially addressing the shortage of skilled labor. Although large language models can help in this scenario, there may be risks, such as generating unreliable or 'hallucinated' content, which could lead people and companies to make bad decisions. Our study proposes integrating human experts into the validation process as a crucial step toward ensuring the proper implementation of technical and administrative controls. Furthermore, we sought to identify how large language models perform in assessing cybersecurity risk scenarios compared to human experts, highlighting the importance of integrating humans and machines in the cybersecurity risk assessment process. Using a questionnaire with risk scenarios, we analyzed responses from 50 human experts. We compared their responses with those of five popular large language models to determine whether it is possible to use only large language models for cybersecurity risk assessment. The results reveal that the large language models consistently underestimated cybersecurity risks compared to human experts, reinforcing the need for human oversight and suggesting that LLMs should be used as complementary tools rather than standalone assessors.

3.8CRFeb 13, 2021
Towards reliable and transparent vaccine phase III trials with smart contracts

Ivan da Silva Sendin, Rodrigo Sanches Miani

Transforming a vaccine concept into a real vaccine product is a complicated process and includes finding suitable antigens and regulatory, technical, and manufacturing obstacles. A relevant issue within this scope is the clinical trial process. Monitoring and ensuring the integrity of trial data using the traditional system is not always feasible. The search for a vaccine against the coronavirus SARS-CoV-2 illustrates this situation. The scientific credibility of findings from several vaccines' clinical trials contributed to distorted perceptions concerning the benefits and risks of the drug. This scenario is ideal for applying technologies such as Blockchain and Smart Contracts in healthcare issues. This paper proposes a protocol based on Smart Contracts, named VaccSC, to enable transparency, accounting, and confidentiality to Phase III of vaccine experiments. The protocol was implemented in Solidity language, and results show that the VaccSC enables double-blindness, randomization, and the auditability of clinical data, even in the presence of dishonest participants.

5.2CRNov 5, 2020
Evaluating the Performance of Twitter-based Exploit Detectors

Daniel Alves de Sousa, Elaine Ribeiro de Faria, Rodrigo Sanches Miani

Patch prioritization is a crucial aspect of information systems security, and knowledge of which vulnerabilities were exploited in the wild is a powerful tool to help systems administrators accomplish this task. The analysis of social media for this specific application can enhance the results and bring more agility by collecting data from online discussions and applying machine learning techniques to detect real-world exploits. In this paper, we use a technique that combines Twitter data with public database information to classify vulnerabilities as exploited or not-exploited. We analyze the behavior of different classifying algorithms, investigate the influence of different antivirus data as ground truth, and experiment with various time window sizes. Our findings suggest that using a Light Gradient Boosting Machine (LightGBM) can benefit the results, and for most cases, the statistics related to a tweet and the users who tweeted are more meaningful than the text tweeted. We also demonstrate the importance of using ground-truth data from security companies not mentioned in previous works.