Memory usage in linux always seems high, regardless of what applications you
are running and if you restarting them or now. And if you leave it as it is,
it raises over time without any reason.

There is a simple explanation : this memory is being used by the system.

This high memory usage is caused by system itself. When there are free
resources, Linux uses them for temporary data, in this situation disk caches.
After system start it would be only applications that use memory, then over
time disk caches usage will be added on top when system is putting cached files
into memory for speedier access.

Application takes priority for that memory, so if there’s need for it to be
used, caches are being dropped quickly and memory is allocated for
applications. Therefore, this is not an issue

To check this on your system, use the following commands :

free -m
echo 3 | tee /proc/sys/vm/drop_caches
free -m

Second run of free should reveal a lot less memory usage if your system is
doing reads/writes.