• Skullgrid@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    arrow-down
    1
    ·
    4 days ago

    this is why I at least sanitise my inputs before using ddg’s ai stuff when I’m shit out of luck and can’t understand the documentation.

    Because it’s just another privacy violating hole.

    I *TRIED* to get a local model working (qwen2.5:1.5b, qwen2.5-coder:1.5b, it needs tooling and shit to be anywhere near useful, and the non coder version hallucinated files that don’t exist). My 8 year old laptop can’t fucking handle the AIs properly.

    Not sure WTF to do about this; in order to be actually useful these things need total context. Maybe some kind of additional submodel (I forget the proper name, LORA or RAG or finetuning or something) to not have it have to try to get me results in the moment.

    As an aside for my street cred, I was fucking around with local Stable diffusion in like Feb 2024 and fucked around with claude a bit back then too.

    Largely not worth it back then, and now, it needs shitloads of processing power to do anything.

    The tool is OK; if you don’t know something/have a skill it can help you achieve *something* but JESUS CHRIST the way they are using it is too fucking obscene.

      • Skullgrid@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        2 days ago

        … Why the fuck is there telemetry on my local AI?? I tried Qwen3.5:2b

        spoiler
        slot launch_slot_: id  0 | task 17 | processing task, is_child = 0
        slot   operator(): id  0 | task 17 | new prompt, n_ctx_slot = 8192, n_keep = 4, task.n_tokens = 439
        slot   operator(): id  0 | task 17 | cached n_tokens = 258, memory_seq_rm [258, end)
        slot init_sampler: id  0 | task 17 | init sampler, took 0.15 ms, tokens: text = 439, total = 439
        [GIN] 2026/10/11 - 16:08:03 | 200 |     324.081µs |       127.0.0.1 | GET      "/api/ps"
        [GIN] 2026/10/11 - 16:08:03 | 200 |       71.38µs |       127.0.0.1 | GET      "/api/ps"
        [GIN] 2026/10/11 - 16:08:03 | 200 |     164.616µs |       127.0.0.1 | GET      "/api/ps"
        [GIN] 2026/10/11 - 16:08:03 | 200 |      96.902µs |       127.0.0.1 | GET      "/api/ps"
        [GIN] 2026/10/11 - 16:08:04 | 200 |      64.533µs |       127.0.0.1 | GET      "/api/ps"
        time=2026-10-11T16:08:04.525-03:00 level=WARN source=types.go:1035 msg="invalid option provided" option=_capturingTokenCorrelationId
        time=2026-10-11T16:08:04.530-03:00 level=WARN source=types.go:1035 msg="invalid option provided" option=_otelTraceContext
        time=2026-10-11T16:08:04.530-03:00 level=WARN source=types.go:1035 msg="invalid option provided" option=_telemetryTurn
        [GIN] 2026/10/11 - 16:08:04 | 200 |  105.201776ms |       127.0.0.1 | GET      "/api/ps"
        time=2026-10-11T16:08:04.801-03:00 level=INFO source=sched.go:1338 msg="updated VRAM based on existing loaded models" gpu=0 library=Vulkan total="2.2 GiB" available="2.2 GiB"
        
        • Womble@piefed.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Those apis are on 127.0.0.1, that’s a shorthand for “this local computer”. This is just different bits of the program communicating with each other.

          You can see that its not going over the internet as the responses are coming back in microseconds.

          • Skullgrid@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            2 days ago

            I mean, the 127.0.0.1 makes sense, but It could be it being sent to that port from VS code… why does my local ollama have any kind of telemetry? is it for network users or if I serve to outside my local machine, so telemetry being sent to me? I think it’s disabled anyway.

            • Womble@piefed.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              1 day ago

              Yep, you can use it to serve models over a network, it essentially creates an api withe the same structure as openai’s ones then serves that. By default just to your machine for other programs to use, but I serve mine to my tailnet so I can use my desktop on my laptop or phone.