You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
complement and subtract are very slow with large number of features #120
The inner working of complement/subtract includes multiple bioframe/pandas operations. Potentially, it can be replaced by something more low-level and faster.
My user experience: subtracting 700_000 regions from 300_000 genomic segments takes 1 hour and 40 minutes to subtract.
Subtract operation when performed for larger numbers of intervals takes longer time. Execution time while subtracting 700_000 regions from 300_000 genomic segments total is found to be less than a minute.
pandas.DataFrame.sort_values is responsible for the most of the execution time which is being internally called via overlap function
Optimising pandas.DataFrame.sort_values seems to be out of scope at this point.
Activity
smitkadvani commentedon Mar 26, 2024
Summary
Experiement
Reproduced subtract operations for large interval.Behaviour of the code is verified using profiling tool.
make_random_intervals function source code is derived from performance-guide documentation for bioframe
Following is the block of profiling stat.
Profiling summary