I’ve tried coding and every one I’ve tried fails unless really, really basic small functions like what you learn as a newbie compared to say 4o mini that can spit out more sensible stuff that works.

I’ve tried explanations and they just regurgitate sentences that can be irrelevant, wrong, or get stuck in a loop.

So. what can I actually use a small LLM for? Which ones? I ask because I have an old laptop and the GPU can’t really handle anything above 4B in a timely manner. 8B is about 1 t/s!

  • shnizmuffin@lemmy.inbutts.lolEnglish
    0·
    20 hours ago

    Let’s take a look at the Developer Agreement that you cited:

    You must only retain chat logs as long as necessary for the operation of Your Services or to improve Your Services; do not do so for the purpose of creating public databases or websites, or, in general, to collect information about Twitch’s end users. You must enable, and process, all requests by end users to block, discontinue, delete, or otherwise opt-out of any retention of chat logs for Your Services.

    This very clearly states that you are disallowed from retaining chat logs for the general purpose of collecting information about Twitch’s end users.

    You said that you, “store ‘facts’ about specific users so that they can be referenced quickly,” but then later in a different thread state, “I’m not storing their data. I’m feeding it to an LLM which infers things and storing that data.” You’re retrieving information about specific users at a later time. You’ve built a database of structureless PII from chat logs. You’ve chosen to store the data as inferences, which makes it a bad database, but still a database.

    I have questions:

    When your streamer mentions something deeply personal, like, “how their mothers surgery went,” that your tool helped them remember, do they disclose that your tool was involved in that transaction? When the viewer gets weirded out and asks your streamer to not mention that again, or forget it entirely, do you have a way to remove that information from your database and a way to prove it’s been deleted? When other people in chat think it’s gross, and ask to opt-out, can you even do it?

    Regarding FrostyTools: I don’t think it’s storing the chat logs for a later time. They don’t have a data retention section in their TOS or Privacy Policy that isn’t related to the streamer. (As in, they hold on to the streamer’s Twitch account and some other information for billing, authentication, etc.) I think it’s taking the chat logs only for as long as it needs to output a response and then deleting it. Also, this excerpt from the FrostyTools TOS made me chuckle:

    This means that you, and not FrostyTools, are entirely responsible for all Content that you upload, post, email, transmit, stream, or otherwise make available via the Service. FrostyTools does not control the Content posted via the Service and, as such, does not guarantee the accuracy, integrity or quality of such Content. You understand that by using the Service, you may be exposed to Content that is offensive, indecent or objectionable. Under no circumstances will FrostyTools be liable in any way for any Content, including, but not limited to, any errors or omissions in any Content, or any loss or damage of any kind incurred as a result of the use of any Content posted, emailed, transmitted, streamed, or otherwise made available via the Service.

    You agree that you must evaluate, and bear all risks associated with, the use of any Content, including any reliance on the accuracy, completeness, or usefulness of such Content. In this regard, you acknowledge that you may not rely on any Content created by the Service or submitted to the Service.

    This leads me to believe that you can violate the Twitch TOS quoted above using FrostyTools. It is apparent that FrostyTools has positioned itself as an application that creates User Generated Content (like Photoshop or Word).