[Feature request] Simple option to set process affinity as a number of cores to use #447
Description
I have Intel Core i7-12700K (on Windows 10), it has 8 main «Performance» cores with hyperthreading, and 4 energy «Efficient» cores, giving in total 16+4=20 virtual cores.
The problem is, if I just run koboldcpp.exe as-is, then after some time, Windows will put its background process to 4 efficient cores (last ones, from 17 to 20). Yes, it is putting all threads of the process from main 16 cores to just 4 efficient cores, no matter how many threads I'll set.
Performance is awful until I put the console window to the foreground, and then boom – all main cores will have 100% load with coolers instantly speeding up!
The solution is to set "process affinity" (for example in Task Manager) for the koboldcpp process, leaving only 16 first cores for it.
Or to start the executable with .bat-file with something like start "koboldcpp" /AFFINITY FFFF koboldcpp.exe
But now I think that other people might have this problem too, and it is very inconvenient to use command-line or task manager – because you have such great UI with the ability to load stored configs!
You can add an option, named something like "max number of cores to use:", which (if not zero) should set the process affinity mask to this number of cores, starting from 0 core. (I believe, efficient cores are always at the end, right?)
You can make a tooltip explaining that it is beneficial to put there a number of "powerful" virtual cores to increase performance; or to specifically limit the used cores by koboldcpp to leave them for other CPU-intensive applications (which plays nicely along with limiting thread count that you already have in GUI).
Do not add full affinity mask support, because most users would not understand how to set it; while those who can – they as well can start in command-line with any desired affinity.