ijeff@lemdro.id to Technology@lemmy.worldEnglish · 1 year agoLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comexternal-linkmessage-square1fedilinkarrow-up162arrow-down14file-text
arrow-up158arrow-down1external-linkLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comijeff@lemdro.id to Technology@lemmy.worldEnglish · 1 year agomessage-square1fedilinkfile-text
minus-squarekorewa@reddthat.comlinkfedilinkEnglisharrow-up3·1 year agoDang I need to try these for now only the stable diffusion extension for automatic 1111 is available. I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram. If it only accelerates 13b then it was already fast enough
Dang I need to try these for now only the stable diffusion extension for automatic 1111 is available.
I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram.
If it only accelerates 13b then it was already fast enough