HybridCodeAuthorship

🎉Published at LREC 2026.🎉 Read the paper here.

HybridCodeAuthorship is a benchmark dataset designed for line-level and chunk-level AI-generated code authorship detection. With the rapid adoption of AI code assistants, industry codebases are increasingly becoming a hybrid of AI- and human-authored code. This dataset simulates the authentic utilization of AI code assistants by interleaving human and AI-authored lines of code.

Dataset Overview

• Base Data: Derived from CodeSearchNet, focusing on Python code files.
• Models Used: Llama 3.3-70B, Llama-4-Scout, and GPT-OSS-120b.
• Size: 10,488 records derived from 4,196 Python code files.
• Total Lines of Code: 2,827,938 (17% AI-generated, 69% nontrivial).
• Quality Checks: Includes labels indicating whether the code passed unit tests or is AST parsable.

Repository Structure

c1-hybrid-code-authorship/
├── data/
│   └── HybridCodeAuthorship.csv
├── LICENSE
└── README.md

Schema

The dataset schema (HybridCodeAuthorship.csv) includes the following columns:

Column Name	Data Type	Description
ModelId	String	Identifier for which LLM was used for code generation.
RecordId	String	A unique identifier for each code sample, serving as the primary key when combined with ModelId.
Language	String	The programming language of the code sample (Python).
GitHubUrl	String	The URL to the original GitHub repository and file.
HumanCode	String	The original, unmodified human-authored code.
HumanCodeTier	String	The validity tier of the human-authored code ("Unit Test Passed", "AST Parsable", "Unparsable").
AICode	String	The final code containing interleaved AI-generated content.
AICodeTier	String	The validity tier of the AI-generated code ("Unit Test Passed", "AST Parsable", "Unparsable").
AICodeLines	List of String	AICode string as a list with each line its own item.
LineNumber	List of Int	List of line numbers for AICode. Used as index for lists in AICodeLines, Attribution, and Triviality columns.
Attribution	List of String	List of attribution labels for AICode lines ("AI" or "Human").
Triviality	List of String	List of triviality labels for AICode lines ("Trivial" or "Nontrivial").
AILineProportion	Float	The actual proportion of lines attributed to AI in the AICode.

Quick Start

Below is a quick demo using Python and pandas to load and inspect the dataset:

import ast
import glob
import pandas as pd

# Reads all chunks and fuses them into one DataFrame in memory automatically
file_paths = sorted(glob.glob("data/HybridCodeAuthorship_part_*.csv"))
df = pd.concat((pd.read_csv(f) for f in file_paths), ignore_index=True)
print(f"Dataset loaded! Shape: {df.shape}")

# Safely evaluate list columns from strings
list_cols = ['AICodeLines', 'LineNumber', 'Attribution', 'Triviality']
for col in list_cols:
    df[col] = df[col].apply(lambda x: ast.literal_eval(x) if pd.notna(x) else x)

# View basic statistics
print(f"Total records: {len(df)}")
print(f"Models used: {df['ModelId'].unique()}")

# Inspect the first record's code and attribution line-by-line
first_record = df.iloc[0]
for line_num, code_line, attribution in zip(first_record['LineNumber'], first_record['AICodeLines'], first_record['Attribution']):
    print(f"[{attribution}] Line {line_num}: {code_line.strip()}")

Intended Use & Limitations

This dataset is intended for researchers and practitioners developing and evaluating fine-grained (line-level and chunk-level) AI-generated code detection algorithms.

Known Limitations:

• Language Scope: The current version is limited to Python code.
• Temporal Scope: To guarantee the absence of AI-generated code in the baseline human samples, the source files from CodeSearchNet are 6+ years old. Consequently, the dataset may not reflect recently developed or popularized Python libraries.
• Pipeline Attrition: Some highly complex or very long code files failed the automated code-interleaving pipeline due to LLM context limits or instruction-following failures, which may slightly impact the macroscopic representativeness of the dataset.

Contributing & Issues

If you discover formatting errors, broken links, or issues with the dataset, please open an issue in this repository. We welcome feedback and community validation!

Citation

If you use this dataset in your research, please cite our LREC 2026 paper:

APA: Patterson, L. S., Wang, L., & Faulkner, A. (2026). HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection. In Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) (pp. 1520–1532). European Language Resources Association (ELRA). https://doi.org/10.63317/4edsbxrqe8na

MLA: Patterson, Luke S., et al. "HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection." Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), European Language Resources Association (ELRA), 2026, pp. 1520-1532. https://doi.org/10.63317/4edsbxrqe8na

IEEE: L. S. Patterson, L. Wang, and A. Faulkner, "HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection," in Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain, 2026, pp. 1520-1532. doi: 10.63317/4edsbxrqe8na

BibTeX:

@inproceedings{patterson-etal-2026-hybridcodeauthorship,
  title = {HybridCodeAuthorship: A Benchmark Dataset for Line-Level Code Authorship Detection},
  author = {Patterson, Luke S. and Wang, Li and Faulkner, Adam},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
  month = {May},
  year = {2026},
  pages = {1520--1532},
  address = {Palma, Mallorca, Spain},
  publisher = {European Language Resources Association (ELRA)},
  editor = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
  doi = {10.63317/4edsbxrqe8na},
  abstract = {Thanks to the rapid adoption of AI code assistants powered by large language models (LLMs), industry codebases are, increasingly, a hybrid of AI- and human-authored code. For risk management and productivity analysis purposes, it is crucial to enable fine-grained location detection of AI-generated code. To develop algorithms for this task, quality benchmarks are needed to assess performance. However, existing benchmarks tend to comprise academic, LeetCode-style problems and presume a code snippet is either completely human-authored or completely AI-authored, which is not reflective of the diverse intents and styles of industry codebases utilizing AI code assistants. To fill these gaps, we introduce HybridCodeAuthorship, a novel benchmark of Python code files with interleaved human- and AI-authored lines of code to simulate authentic utilization of AI code assistants. In this paper, we first present our dataset construction pipeline, which leverages CodeSearchNet, a massive collection of links to open sourced repositories on GitHub. We then benchmark the performance of two state-of-the-art AI-generated code detection algorithms at both the line- and chunk-level. Experimental results demonstrate that HybridCodeAuthorship is a challenging benchmark with a top-scoring algorithm, AIGCode Detector, obtaining a highest F1 score of 0.48 and 0.56 on line-level and chunk-level code detection tasks, respectively.}
}

License

This project is licensed under the Apache 2.0 License.

Authors

Luke Patterson, Li Wang, Adam Faulkner (Card Intelligence, Capital One).