This is a quick explanation of how you can improve performance of the vstserver when launched and used by a normal user.
About user mode and RT
Linux musical applications need to gain realtime (RT) priority to achieve best performance. This RT mode is usualy only granted to the super user (root) for security reasons, but there are some workarounds.
If your are not too exposed (i.e., you are the only user of your machine and off the network) you can make an application suid root by doing 'chmod +s' on the application executable, but this method won't work with vstserver since it relies on WINE. The proper way would be to suid root WINE instead, but that would create problems of access for the temporary files created and used by vstserver. Fortunately there is another method which is much more secure: a kernel feature called "capabilities".
First you need to use a kernel that supports capabilities. If you run a multimedia kernel provided by Planet CCRMA, AGNULA, Thac's Mandrake, or some other distro it is very likely that you have already everything. If not, you need to apply a very small patch to one of the kernel source files and then recompile your new capabilities-enabled kernel. Next you need to have Tommi Ilmonen's givertcap installed on your system, which in turn requires the capabilities library (libcap is included with most mainstream Linux distros).
If you don't already have givertcap installed grab the source package from the givertcaps homepage and follow the simple installation instructions also found there.
Givertcap will be installed in /usr/local/bin by default, but currently vstserver searches for it /usr/bin so either move the binary or make a symbolic link to it there. The author of vstserver will change this soon so that a variable will need defined to specify the right path. Check the latest documentation of vstserver for possible changes.
And that's it. Now you can start vstserver as a normal user and vstserver will call givertcap to gain RT mode for your VST plugins. Now when you instantiate a VST plugin you should see these messages in the console:
trying -/DATA/windoz/VstPlugins/NorthPole.dll- The process 1583 was give capabilities = cap_ipc_lock,cap_sys_nice,cap_sys_resource+eip VSTSERVANT/PROCESS_thread: Realtime priority set for "NorthPole.dll". The process 1585 was give capabilities = cap_ipc_lock,cap_sys_nice,cap_sys_resource+eip VSTSERVANT/MAINHANDLER_control: NorthPole.dll started.And no glitches ! :)
Problem: You start vstserver, then MusE, then load the NorthPole (or other) VST plugin on the Audio Master and see this message in the terminal window :
>> trying -/DATA/windoz/VstPlugins/NorthPole.dll- >> sh: line 1: /usr/bin/givertcap: Aucun fichier ou répertoire de ce type (file not found) >> VSTSERVANT/PROCESS_thread: Unable to set SCHED_FIFO realtime priority for "NorthPole.dll". >> VSTSERVANT/PROCESS_thread: Unable to lock memory for "NorthPole.dll". >> VSTSERVANT/MAINHANDLER_control: NorthPole.dll started.The plugin will still work but you will have glitches ! :(
Solution: It means that givertcap hasn't been found (move it in /usr/bin), or simply is not installed. See the Capabilities section above.