Support for SermonSpeaker 5.x is in bugfix mode only, the same as it is for Joomla 3.10.
I will not add any new features to this releases.
Please upgrade to SermonSpeaker 6.x and Joomla 4.x, which is better anyway.
Unfortunately this seems to be a bug. SermonSpeaker tries to find a matching menu item for each link, taking the first match found. But sometimes this fails. I'll have to see if I can figure out what happens here.
It looks like I need to add some checks for the selected category and also check the current menu item.
For now your only option would be to create hidden menu items each for this particular sermons. Then SermonSpeaker should find those menu items as a better match.