Of course, segulot, brachot, the words of tzaddikim, all have no value. The world is a random place, and we are at it's mercy, though the strength of our arm (or brain) may help a bit. Glad I don't live in your world. It's a pretty hopeless place.
As Misnagid has already pointed out, Akiva's comment fails as an argument, because the consequences of a belief have no bearing on the veracity of the belief itself. The fact that a world without sugulot makes Akiva sad is no proof that segulot exist.
But let's play ball, and examine his assertion for a moment.
If segulot are real, Judaism is pretty easy. After all, if mundane objects like red strings or havdalah wine can be imbued with power and promise, religious practices become redundant, or even unnecessary. Why bother perfecting your character and intellect via the hard work of mitzvot, if a magical object can provide protection and blessing? Think about it: Is it justice to reward someone for embracing a segula irrespective of his other merits?
As you might expect, the Rishnoin agreed segulot were nonsense. The Rambam goes on for paragrphs about it in the Moreh, and the Radak also fumilates:
"...people thought that many things and certain actions would help or damage, sicken or heal. These things are not based upon wisdom or medicine or natural forces, but rather they are based upon the repeated customs of people throughout the generations...and these are the darkei ha-emori." (Radak on Yeshayahu 40:21)
I hope Akiva, and others, will recognize that segulot have no affect on our lives, or on reality. Reward and punishment is a function of merit, not magic.