Possible bug with the "Gather the mysterious fragments" step of the quest and a potential fix. (Spoilers I guess, but I'm not going to talk about story at all)

I was about to post asking for help, but fixed it myself, so I'm now going to share how I fixed it in case anyone else has the same problem.
Hovering over the top 3 fragments doesn't light up, and releasing doesn't gather them. It works fine on all others, just not these three. I've seen others say the same but with a different three.

After a bit of messing around, I got it to work. I'm not sure what fixed it, so here's everything I did between the last time it didn't work and the first time it did.

1. Held alt to push the gather button manually, which made me change directions to face the camera (which was backwards).
2. tried and failed to connect a controller (no options menu to swap control settings)
3. walked around with the character and came back to the spot to attempt again.

One of those things might have been what fixed it, and my money's on the moving around. Maybe it was something to do with the angle I was facing and my distance to the interaction?
Either way, hopefully this helps if anyone else can't get passed this point.