Life's hectic. Let's get funding for the big one: https://github.com/stylehouse which is a direct attack on the display problem of the computer. Lately I think we should use cytoscape in a big way! We can make stuff float out of stuff on tendrils. This is the reality of things. Anyway...

A wee side project now: https://github.com/stylehouse/jamsend which integrates a bunch of tech I need for stylehouse to be an internet phenomenon, but also might get impressive (for funding) fast enough to nice up my life, because I need it.

This webapp will, soon, use the Directory API to turn your local music pile (and that of your peers) into your own private Napster!

Or (completely different UI) a radio station (or spectrums of), which you can share with your old people. a bit of a DJ system maybe.

It's hard to tell how complex to go. Layers are:

p2p via off-the-shelf abstractions on WebRTC
pubkeys and trust enabling certain other features
Any Other Features (eg directory sharing, music streaming)

That Any is likely to explode a bit. Surely we need to set some sort of data layer standard, or should we just leave it as sheer svelte? It's like leading a flock of followers using your technology - you've got to set the vibe.

Lets begin:

Beginning

Lets use the Web Crypto API to verify the stream distributing through a p2p network.

But then, use ed25519 instead, because:

available Web Crypto seems to be End-Of-Life-ing (some say) at the end of this decade (5 years away)
the pubkeys are much shorter... Of course, you don't need to give out the whole pubkey in a URL, just enough to prevent hash collisions (pubkeys are derived unpredictably), so that someone in the room can claim to own the room, etc.

    import * as ed from '@noble/ed25519';
    import { sha512 } from '@noble/hashes/sha512';
    ed.etc.sha512Sync = (...m) => sha512(ed.etc.concatBytes(...m));
    import QrCode from "svelte-qrcode"
    import SvelteCopyUrlButton from 'svelte-copy-url-button';
    import { untrack } from 'svelte';
    import { Idento } from './Peer.svelte';

Explains a lot
A static page
may have an identity to find in the URL

And|or you begin a place!

And...

Then I start to gather peers, but using Trystero (modern-but-perhaps-2016-looking WebRTC p2p party plumbing) with the bittorrent peer finding strategy seems flaky and slow, so it's mqtt during development -> needs crypto.subtle... -> needs TLS -> my public Caddy server (d) needs a new... port? to put this other app over?

Hoping to make a Minimum Viable Product this week, I swap the idea of peer-sourced media for an app server with "demo music" on it via websocket. We can gradually swap in peer-sourcing and p2p-transit some time later.

Then, the vite dev server's other websocket connection to the client, which it uses to notify of code changes (hot module reload, HMR), is closing nearly a second after the page loads, which spurs a location.reload! So it's tailspinning.

And stays that way, until I swap bun for npm, then turns into a "idealTree already exists" error, while doing npm i.

What the hell does that mean? This is bad, in retrospect it could have complained about a lack of package.json before pretending to be all good, and I doubt idealTree already exists...

Well apparently, it'll be working in (aka WORKDIR) /, in the container's short-lived overlay filesystem (per run), whereas we want it working in /app, where we've got this mount... Anyway, the readme should say:

setup

First time,

docker compose build
docker run -v .:/app jamsend-jamsend-dev bash -c "cd /app && npm i"

Thence,

docker compose up

And this took all goddamn day!

Then!

// a unique song URI hash
export type urihash = String

// sometimes requests for more audio have specifics:
export type audioi = {
    // identifies a track
    id: urihash,
    // position they're streaming towards
    index: Number,
}
// the response of part|whole
type audiole = audioi & {
    blob: Uint8Array,
    // < last bit -> start another from the start
    done?: Boolean
}

I don't know how it's going to go decoding these blobs that are just chopped pieces of music files.

They (decoders) tend to skip forward until some kind of frame, but how would it even know the format of the stream, which might vary...

Tomorrow's trickiness!

More here soon.

And there was more...

But then the editor lost my write up! How can I be proud of my work on such a machine?

And then loss happened again! I had described getting to...

Prod

It isn't obvious how to include the socket.io part of the server with the svelte|vite build. It doesn't just happen for prod. In dev, with vite, the webSocketServer was given in vite.config.ts.

It turns out you have to bundle an extra webserver around it as a separate build (the build:server action in package.json), that becomes the top-level and includes the other|inner build|webserver as such see: https://github.com/stylehouse/jamsend/commit/40f5d856b214d278b9fa3305a6666329d47be11c

Then the prod-ing process itself is just a pull and a couple of configs changes: https://github.com/stylehouse/jamsend/commit/df7a361e0dff33808b8aed6a41fa554923910fe1

There was this last minute bug I got around to:

ffloudness timeout, SIGINT, still get some output

this helps a lot with diversity in the first 30 seconds

or lots of aud wait for ffloudness to etc etc

https://github.com/stylehouse/jamsend/commit/b2da348062e8e967b5b57ce1ea76e0cb5c61af5a

Anyway, the same way we have our dev instance on https and visible to the world, via Sydney, for $6.9 per month:

We now have website like:

Which is totally going to get me in trouble for giving out music. Those guys on the drums. That music might be owned by a hedge fund now, and have people actively scrubbing their property off other people's property. It could get messy.

This project has now been two months and produced 4000 lines of non-strict typescript!

If we want to keep enjoying this growth:

We'll have to outsource to...

Peering

Something I've already got! I'm torn between putting it straight in here, and fixing jamola a bit more...

I think I'll just go for it. Back soon.

Back. Things are just barely p2ping now:

Is this playing out:

    handlers = {
        hello: (data) => {
            console.log("they say hi: ",data)
            if (!this.said_hello) {
                this.say_hello()
            }
            else {
                this.emit('story',{yadda:3})
            }
        },
        story: (data) => {
            console.log("they say story: ",data)
            data.yadda++ < 9
                && this.emit('story',{yadda:data.yadda})
        },
    }

I went for PeerJS rather than something Claude talked me into building myself.

It was one of the big fuckups of last year, not drilling all the way through to PeerJS. I have no idea why I failed to use it for jamola, it wasn't that far away... I also tried Trystero again but it seems to be loads of trouble with modern bundlers. There was a big wave of javascript culture around 2014, and now in 2025 some significant-looking projects could be integration hell. Projects talking about script tags instead of imports is a sign of age.

Sysadminy

Various details now about how to run your own peerjs-server behind Caddy, because I got some arguments wrong somewhere and mistook the whole situation for getting rate-limited by 0.peerjs.com ...:

leproxy Caddy is now configured to:

    if ($name =~ /^d?(jam|vou)/) {
        # share a WebRTC signaling server aft Caddy
        $extra .= <<"";
          handle_path /peerjs-server/* {
              reverse_proxy $host:9995
          }

    }

    print $fh <<""
      $name.duckdns.org {
          encode zstd gzip

    . $extra
    . <<"";
          handle {
            ...

that's a Caddy config in a docker compose being made by a perl script.

It takes some of our traffic over here:

  peerjs:
    image: peerjs/peerjs-server
    container_name: jamsend-peerjs
    ports:
      - "172.17.0.1:9995:9995"
    command: ["--port", "9995"]
    # the Caddy handle_path in leproxy takes this bit of path off
    #, "--path", "/peerjs-server"]
    restart: 'always'

which is in the jamsend docker compose, incidentally. So leproxy knows about peerjs-servers, but only if they've got those kinds of names...

from here:

export class Peerily {
    ...
    async listen_pubkey(pub) {
        ...
            eer = this.setupPeer(pub)
            this.addresses.set(pub,eer)
            this.address_to_connect_from ||= eer
...    }
    setupPeer(pub:prepub) {
        // these listen to one address (for us) each
        let eer = new Peer(this, pub, {
            host:location.host,
            port:443,
            path:"peerjs-server"})

Etc!

Now, diring... Got so slapped by work today. Sheesh. Back in a mo.

Peerily

Turns out to need a binary-safe emit(), which PeerJS provides, but also we want to sign everything:

    async emit(type,data={},
        ...
            // assures everything we say
            let crypto = {}
            crypto.sign = enhex(await this.P.Id.sign(json))
            if (buffer) {
                crypto.buffer_sign = enhex(await this.P.Id.sign(buffer))
            }

So we've made every emit() into 2-3 send()s. The signatures, the signed data, and perhaps signed binary data.

    async unemit(data:any) {
        ...
        else if (this.next_unemission == 'data') {
            if (typeof data != 'string') throw "not string"
            let crypto = this.next_unemit.crypto
            // check it's them
            if (this.Ud) {
                let valid = await this.Ud.verify(dehex(crypto.sign), data)
                if (!valid) throw `invalid signature`
            }
            this.next_unemit.data = JSON.parse(data)

So now we have a connection-maker that we can't seem to kick over - it retries!

Peerily.svelte.ts is now useful and elegant!

Going forward to Wednesday, I expect:

 we
  jamsend
   < who we might know
      what they name themselves
       how we push their name around
        how we push or define anything
        %sc
      trust with tags
       trust things they say we said before
        eg feed audio in
        this way around saves space on popular hosts
         people bring back their hi scores etc
     tags enable sets of commands (msg.type)
   < cytoscape music explorer
     run by %sc pooling
     
   < then doing a data share
   < transparently passing bits of filesystem along
   
  or:
   < noticing webrtc connections falling down
   < swarm health chatter
   

Remembering

But first! Javascript that doesn't complain about rogue variables now?

        let on_disk = disk_piers?.[0]?.[stashedkey]
        let ok = in_situ == is && on_dist == is

This on_dist was red-underlined, by typescript in vscode, and simply evaluates to undefined.

Bizarre. The art of catering to the moment when something's gotta blow up in someone's face a little bit about something...

Anyway. We needed to test remembering, because it:

- wasn't working too good...

- iterating on it involves testing labour: refreshes, UI interaction, and logging objects into DevTools to Expand Recursively

On the side of our project, I added a src/routes/repro-reactive-stashed-hierarchy/+page.svelte route, which includes the test component src/lib/p2p/ui/repro-reactive-stashed-hierarchy/Top.svelte, and the test case starts like this:

    let P:NotPeerily|null = $state()
    async function top() {
        console.warn(`Starting test case...`)
        this_test_number = this_test_number + 1
        stashed = "{}"
        OKs = []
        P = new NotPeerily({save_stash})
        P.startup()
        await wait()
        nudge()
        await wait()
        eer_ok('leg',3,'pier creates stashed properties')

That begins to:

export class NotPeerily {
    stash = $state({})
    save_stash:Function|null
    constructor(opt={}) {
        Object.assign(this, opt)
    }

    // for test convenience, has the first|only one of:
    eer:NotPeering
    pier:NotPier
    startup() {
        this.eer = this.a_NotPeering('one')
        this.pier = this.eer.a_NotPier('two')
    }

Then, a bit of a CRUD for the eer or pier, they're pretty similar:

    // own pubkey addresses
    //  are one per Peer, so we create them here
    addresses:SvelteMap<Prekey,Peering> = $state(new SvelteMap())
    a_NotPeering(id) {
        let eer = this.addresses.get(id)
        if (!eer) {
            eer = new NotPeering({P:this})
            this.addresses.set(id,eer)
        }
        // stash it with our known selves (keypairs, listen addresses)
        let stashed = this.stash.Peerings?.find(a => a.id.startsWith(id))
        if (!stashed) {
            eer.stashed = {id}
            stashed = eer.stashed
            this.stash.Peerings ||= []
            this.stash.Peerings.push(stashed)
        }
        eer.stashed = stashed
        arre(this.stash.Peerings,stashed,eer.stashed)
        return eer
    }

Lots to unpack.

The eer.stashed = {id} is when a proxy object is created, and that is what we then put into stashed.

It must be done on a separate line!

The proxy object goes into the enclosing stashed state, which should get it into this bit of Top.svelte:

    let stashed = $state()
    let save_stash = throttle(() => {
        console.log(`saving Astash`)
        stashed = JSON.stringify(P.stash)
    },100)

So the stash is entirely not-too-much as a graph (since svelte's proxy objects seem to go fine into JSON.stringify), whereas its live-object counterpart addresses is full of objects that might link to way too much stuff. It's easy to log proxy objects into DevTools that will infinitely Expand Recursively. The need for attention is everywhere!

Anyway, for addresses we use SvelteMap makes a reactive thing to feed to the UI in Top.svelte:

{#each P?.addresses as [pub,eer] (pub)}
<NotPeering {pub} {eer} />
{/each}

Which is basic.

The tricky bit was getting changes from this function to this effect:

    async function tweakstash() {
        pier.stashed.leg ||= 2
        pier.stashed.leg = pier.stashed.leg + 1
        pier.stashed.thinke = 3
        console.log(`Pier thinked`)
    }
    pier.tweakstash = tweakstash
    $effect(() => {
        // console.log('Pier stashed changed:', JSON.stringify(pier.stashed))
        if (Object.entries(pier.stashed)) {
            console.log(`Pier stashed save...`)
            pier.P.save_stash()
        }
    })

Just if (pier.stashed) { isn't enough - you have to look for individual things that changed. Or it's just the same stuff in a new container! How basic.

Finding the above tricks took a week off my life. It works:

And it works fine for deep objects, I guess:

    function tweakdeeply() {
        pier.stashed.swan ||= {}
        pier.stashed.swan.of ||= {}
        pier.stashed.swan.of.did ||= 2
        pier.stashed.swan.of.did++
    }

These experiments are on this branch: https://github.com/stylehouse/jamsend/tree/repro-reactive-stashed-hierarchy

Now to take these learnings back to the app itself!

Sharing

Is coming...

Hows Ecology Squad

Wednesday, 14 May 2025

jamsend - a simplification