Skip to content

Fixit is incredibly slow with many files. #458

@jarshwah

Description

@jarshwah

Related to #457 I was doing some profiling on our custom fixit rules, and determined that the number of files being linted was the primary determinate in fixit runtime.

To that end, I've setup a repository that demonstrates the performance issues. It contains a large number of empty __init__.py files in a tree of modules, about 41,000 files in total. It has a single fixit rule with a visit_Module that does nothing except return. Running fixit against this tree pegs all of my cores to 100% and takes about 40 seconds to complete.

https://github.com/jarshwah/experiments-fixit is the repository which I'm using to document my findings. So far I've only looked at the file-finding, but plan to dig in further where time permits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions