SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity
Authors: Pengfei Jing, Mengyun Tang, Xiaorong Shi, Xing Zheng, Sen Nie, Shi Wu, Yong Yang, Xiapu Luo | Published: 2024-12-30 | Updated: 2025-01-06
LLM性能評価
Safeguarding System Prompts for LLMs
Authors: Zhifeng Jiang, Zhihua Jin, Guoliang He | Published: 2024-12-18 | Updated: 2025-01-09
Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection
Authors: Ira Ceka, Feitong Qiao, Anik Dey, Aastha Valecha, Gail Kaiser, Baishakhi Ray | Published: 2024-12-16 | Updated: 2025-01-18
LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states
Authors: Luis Ibanez-Lissen, Lorena Gonzalez-Manzano, Jose Maria de Fuentes, Nicolas Anciaux, Joaquin Garcia-Alfaro | Published: 2024-11-29 | Updated: 2025-01-10
CleanVul: Automatic Function-Level Vulnerability Detection in Code Commits Using LLM Heuristics
Authors: Yikun Li, Ting Zhang, Ratnadira Widyasari, Yan Naing Tun, Huu Hung Nguyen, Tan Bui, Ivana Clairine Irsan, Yiran Cheng, Xiang Lan, Han Wei Ang, Frank Liauw, Martin Weyssow, Hong Jin Kang, Eng Lieh Ouh, Lwin Khin Shar, David Lo | Published: 2024-11-26 | Updated: 2025-01-16
CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity
Authors: Zhengmin Yu, Jiutian Zeng, Siyi Chen, Wenhan Xu, Dandan Xu, Xiangyu Liu, Zonghao Ying, Nan Wang, Yuan Zhang, Min Yang | Published: 2024-11-25 | Updated: 2025-01-17
On Calibration of LLM-based Guard Models for Reliable Content Moderation
Authors: Hongfu Liu, Hengguan Huang, Hao Wang, Xiangming Gu, Ye Wang | Published: 2024-10-14
Can LLMs be Scammed? A Baseline Measurement Study
Authors: Udari Madhushani Sehwag, Kelly Patel, Francesca Mosca, Vineeth Ravi, Jessica Staddon | Published: 2024-10-14
Decoding Secret Memorization in Code LLMs Through Token-Level Characterization
Authors: Yuqing Nie, Chong Wang, Kailong Wang, Guoai Xu, Guosheng Xu, Haoyu Wang | Published: 2024-10-11
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
Authors: Tingchen Fu, Mrinank Sharma, Philip Torr, Shay B. Cohen, David Krueger, Fazl Barez | Published: 2024-10-11