Took an initial crack at solving the real problem and while I still need to go back and make some changes elsewhere in the program to make this do the right thing all the time, the test cases I've been using to diagnose the performance issue are now finishing in about a tenth of a second instead of taking almost a minute and that's honestly good enough.