It's not necessarily easier because you collect more fragments but more because of the CP you get from collecting certain fragments. So ultimately, it's a higher level. I only had about 116 when I found the final battle easy. The time it took me 40 minutes, I had about 110 fragments.
But...