Memory management in Linux is complex, there where jokes that only Linus Torvalds knows what the kswap service does.
The amount of usage of swap does not indicate the memory usage of your system. In the swap are the memory pages which are not currently used and they stay there as long there is space. Even if you got plenty of ram, over time, your swap fills up. I’m cutting corners here, as said only Linus knows how it really works.
What we really need is a memory plugin for collected; now i’m teasing @giacomo 
see