Consolidate storage, sync, NIP-98, push server, OpenRouter, audio, call and title helpers in src/app and src/lib

This commit is contained in:
Jon Staab 2026-09-26 00:30:16 -07:00
parent 5343a28e6c
commit c8c6bd4465
36 changed files with 878 additions and 994 deletions

View file

@ -114,19 +114,22 @@ imports. `flotilla-views` covers component conventions.
- `classifieds.ts`, `articles.ts`, `pins.ts` (a person's pinned notes), `pinboards.ts`
- `reactions.ts`, `social.ts` (display names, comment trees, muting), `render.ts` (events as
text), `statuses.ts` (NIP-38), `uploads.ts` (Blossom, image compression)
- `notifications.ts` (unread state, badges), `inbox.ts` (the home inbox)
- `notifications.ts` (unread state, badges, the home inbox)
**Messaging and calls**
- `chats.ts`, `call.ts` (call state), `callEngine.ts` (LiveKit join, leave, devices, and the
`AbortError`/`TimeoutError` a join rejects with)
- `chats.ts`, `call.ts` (call state), `callEngine.ts` (LiveKit join, leave, devices, the video
tiles a call shows, and the `AbortError`/`TimeoutError` a join rejects with). `call.ts` stays
free of livekit values, since the root layout loads it and `callEngine.ts` is imported lazily.
**Identity and payments**
- `nip46.ts`, `pomade.ts` (email login), `lightning.ts` (wallet, invoices), `healthChecks.ts`
(prompts for missing inbox/outbox relays)
**Platform and voice**
- `push/` (notification adapters), `share.ts`, `deepLinks.ts`, `keyboard.ts`
- `dictation.ts` (speech-to-text) and `speech.ts` (read aloud), both through OpenRouter
- `push/` (notification adapters, and `server.ts` for the push server's HTTP API), `share.ts`,
`deepLinks.ts`, `keyboard.ts`
- `dictation.ts` (speech-to-text) and `speech.ts` (read aloud), both through `postOpenRouter` in
`openrouter.ts`
A feature gets a `src/app/<feature>.ts` only when it has non-UI logic to hold. Polls, goals,
threads and calendar events have no module; their components use domain readers directly.
@ -142,6 +145,8 @@ client.
- `indexeddb.ts`: the `IDB` wrapper that `storage.ts` builds on
- `feeds.ts`: saved feed definitions (kind `FEED`) over `@welshman/feeds`. It is unrelated to
`@app/feeds`, which loads events.
- `audio.ts` (device lists, a live input level, WAV encoding), `tileGrid.ts` (the call grid's
geometry)
- `currency.ts`, `transition.ts`, `implicit.ts` (hands state from one page to the next)
- `test/`: the DEV-only hooks the e2e harness injects through
- `components/`: the design system, entered through `theme.css` (see `flotilla-views`), which

View file

@ -300,7 +300,7 @@ These predate the rule or are waiting on welshman. Don't copy them; fix one when
- **No factory yet:** `MESSAGE` (9) in `RoomChat` and `publishRoomQuote`; 9009 room invites in
`src/app/access.ts`; `LIVEKIT_PARTICIPANTS` (39004, a local constant in `src/app/call.ts`);
`STATUS` (30315) in `src/app/statuses.ts`, where `ProfileStatus` still uses `tags.find`; push
subscriptions (a literal 30390) in `src/app/push/adapters/capacitor.ts`; `DIRECT_MESSAGE_FILE`
subscriptions (`PUSH_SUBSCRIPTION`, 30390, local to `src/app/push/adapters/capacitor.ts`); `DIRECT_MESSAGE_FILE`
(15) in `Chat.svelte`.
- **Factory exists but bypassed:** DMs built with `makeEvent` in `Chat.svelte` (`DirectMessage`);
relay lists in `SignUp.svelte` (`RelayList`, `MessagingRelayList`); deletes in

View file

@ -52,4 +52,4 @@ Gift wraps are only synced once the user opts in (`shouldUnwrap` in `src/app/syn
| Follows and mutes | `FOLLOWS` (3), `MUTES` (10000) | `FollowList`, `MuteList` | `src/app/social.ts` | people pages, muting |
| Relay lists | `RELAYS` (10002), `MESSAGING_RELAYS` (10050), `SEARCH_RELAYS` (10007), `BLOCKED_RELAYS` (10006), `BLOSSOM_SERVERS` (10063) | `RelayList`, `MessagingRelayList`, `SearchRelayList`, `BlockedRelayList`, `BlossomServerList` | plugins in `src/app/core.ts` | `settings/*` |
| Account deletion | `VANISH` (62, written as a literal), `DELETE` (5) | none; `Delete` bypassed | none | `ProfileDelete` |
| Push subscriptions | 30390 (a literal, no constant) | none | `src/app/push/adapters/capacitor.ts` | background |
| Push subscriptions | 30390 (`PUSH_SUBSCRIPTION`, local) | none | `src/app/push/adapters/capacitor.ts` | background |

View file

@ -62,8 +62,8 @@ The following run outside that gate:
- every app policy
This code reads `app.get().user?.pubkey` and bails when it is missing, as
`syncUserSpaceMembership` in `src/app/sync.ts` and `nip98Header` in `src/app/notifications.ts`
do.
`syncUserSpaceMembership` in `src/app/sync.ts` and `syncCheckedRemote` in
`src/app/notifications.ts` do.
Stores derived from `user` throw as soon as they are subscribed signed out. That includes
`isEventMuted` (`social.ts`), `deriveUserIsRoomAdmin` (`rooms.ts`) and `deriveUserCanCreateRoom`
@ -106,7 +106,8 @@ Bookkeeping for one identity lives on a plugin instance, so it is discarded with
`Commands` in `src/app/commands.ts` keeps its `pulled` set on the plugin for this reason.
Module-level caches are fine when their contents do not depend on who is signed in, or when what
they cache is itself a rebinding store. `commandsByUrl` holds `fromApp` stores.
they cache is itself a rebinding store. `deriveCommandsForUrl` (`commands.ts`) caches `fromApp`
stores with `simpleCache`.
`hasBlossomSupport` (`uploads.ts`) and `deriveHasLivekit` (`relays.ts`) use `simpleCache` from
`@welshman/lib` to share one store per url across every component that asks.
@ -272,8 +273,9 @@ Room messages, threads and other content are not kept, and background sync pulls
next tick.
To persist another kind, add it to `kinds` in `storage.ts`. To persist a new map-backed plugin,
add a `TABLES` entry and an `init*` method shaped like `initHandles`: load the rows, subscribe
to `onItem`, and batch the writes.
add a `TABLES` entry and an `init*` method that calls `persistMapPlugin` with the table, the
plugin, and how to turn a row into a key and item and back. It loads the rows, subscribes to
`onItem`, and batches the writes.
### kv, ss and synced stores
@ -346,8 +348,9 @@ call tears down the previous run.
- `syncDMs` pulls gift wraps from the user's messaging relays, only when `shouldUnwrap` is on.
`pullAndListen` is a negentropy `Sync.pull` plus a live `limit: 0` request, both stopped through
an `AbortController`. `syncSpaces` and `syncUserData` diff their `unsubscribersBy*` maps against
the room list, so a new filter goes into the right `pullAndListen` call.
an `AbortController`. `syncUserData`, `syncSpaces` and `syncDMs` each keep one sync per url with
`makeKeyedSyncs`, whose `reconcile(urls)` starts the new ones and stops the dropped ones. A run
reads the pubkey once when it starts, since login triggers a fresh `syncApplicationData`.
Background sync keeps badges, navigation, the inbox and notifications correct on any page. Data
that must be current app-wide belongs here. Data only one page shows is loaded by that page's
@ -415,8 +418,9 @@ NIP-86 calls (`relayManagement.get().forUrl(url)`) are not thunks. They return
## Other app-level stores
- **A join over many sources.** `notifications.ts` derives `latestActivityByPath`, then
`allNotifications`, then `notifications` and the counts. `inbox.ts` derives from the same two
stores, so the inbox matches the badges.
`allNotifications`, then `notifications` and the counts. The inbox stores
(`inboxConversations`, `inboxSpaceContent`) derive from the same two, so the inbox matches the
badges.
- **Singleton session state.** `call.ts` keeps call state in plain writables (`callState`,
`currentCallSession`, …).
- **UI signals.** `toast` in `toast.ts`, and `relaysPendingTrust` in `policies.ts`.

View file

@ -7,8 +7,4 @@
@import "./components/room.css";
@import "./components/space-menu.css";
/* SocketStatusIndicator builds its class as bg-{status.theme}, which the source
scanner can't see. Must stay below every @import to keep those valid. */
@source inline("bg-success bg-warning bg-error bg-gray-500");
@config "../../tailwind.config.js";

View file

@ -9,181 +9,6 @@ import {events, network} from "@app/core"
export const LIVEKIT_PARTICIPANTS = 39004
/** The lower aspect bound is 1:1 on landscape and 3:4 on portrait, and the upper is 16:9. */
const TILE_ASPECT_PORTRAIT = 3 / 4
const TILE_ASPECT_LANDSCAPE = 16 / 9
const TILE_GAP = 8
/** Below this the grid scrolls rather than forcing portrait tiles, which is ~6-8 tiles on a portrait phone. */
const MIN_TILE_HEIGHT = 120
export type TileRow = {
columnCount: number
tileWidth: number
tileHeight: number
/** Total width of this row including gaps — for the template's max-width */
rowWidth: number
aspectRatio: number
}
/** All tiles share one size. Full rows fill the container width and a partial row is centered. */
export type AdaptiveTileGrid = {
rows: TileRow[]
totalWidth: number
totalHeight: number
}
/** Score for comparing candidate layouts. Lower is better. */
type LayoutScore = {
/** Penalty for vertical overflow: 0 if none, else huge */
verticalOverflowPenalty: number
/** Penalty for horizontal overflow: 0 if none, else huge */
horizontalOverflowPenalty: number
/** Penalty when tiles fall below MIN_TILE_HEIGHT even while fitting */
minTileHeightPenalty: number
/** Negative tile area (we want to maximize area, so negate) */
negativeArea: number
/** Leftover whitespace (underflow) in px */
whitespace: number
/** Average absolute deviation from 16:9 across all tiles */
aspectDeviation: number
}
const compareScores = (a: LayoutScore, b: LayoutScore): number => {
const fields: (keyof LayoutScore)[] = [
"verticalOverflowPenalty",
"horizontalOverflowPenalty",
"minTileHeightPenalty",
"negativeArea",
"whitespace",
"aspectDeviation",
]
for (const field of fields) {
if (a[field] !== b[field]) {
return a[field] - b[field]
}
}
return 0
}
/** The largest tile that fits, shrunk to whichever dimension is more constraining. */
const fitTile = (availWidth: number, availHeight: number, minAspect: number) => {
const fillAspect = availWidth / availHeight
const aspectRatio = Math.max(minAspect, Math.min(TILE_ASPECT_LANDSCAPE, fillAspect))
if (fillAspect >= aspectRatio) {
const tileWidth = availHeight * aspectRatio
return {tileWidth, tileHeight: availHeight, aspectRatio}
}
const tileHeight = availWidth / aspectRatio
return {tileWidth: availWidth, tileHeight, aspectRatio}
}
/** A candidate grid for a given column count, with the partial last row centered. */
const buildCandidate = (
tileCount: number,
columnCount: number,
containerWidth: number,
containerHeight: number,
minAspect: number,
): AdaptiveTileGrid | undefined => {
const gap = TILE_GAP
const fullRowCount = Math.floor(tileCount / columnCount)
const remainder = tileCount % columnCount
const totalRowCount = fullRowCount + (remainder > 0 ? 1 : 0)
const availHeight = (containerHeight - (totalRowCount - 1) * gap) / totalRowCount
const availWidth = (containerWidth - (columnCount - 1) * gap) / columnCount
if (availWidth <= 0 || availHeight <= 0) {
return undefined
}
const {tileWidth, tileHeight, aspectRatio} = fitTile(availWidth, availHeight, minAspect)
const rows: TileRow[] = []
for (let r = 0; r < fullRowCount; r++) {
rows.push({
columnCount,
tileWidth,
tileHeight,
rowWidth: columnCount * tileWidth + (columnCount - 1) * gap,
aspectRatio,
})
}
if (remainder > 0) {
rows.push({
columnCount: remainder,
tileWidth,
tileHeight,
rowWidth: remainder * tileWidth + (remainder - 1) * gap,
aspectRatio,
})
}
const totalHeight = totalRowCount * tileHeight + (totalRowCount - 1) * gap
const totalWidth = columnCount * tileWidth + (columnCount - 1) * gap
return {rows, totalWidth, totalHeight}
}
/** Scored on overflow first, then whitespace, then distance from 16:9, then tile size. */
export const computeAdaptiveGrid = (
tileCount: number,
containerWidth: number,
containerHeight: number,
): AdaptiveTileGrid | undefined => {
if (tileCount <= 0 || containerWidth <= 0 || containerHeight <= 0) {
return undefined
}
const minAspect = containerWidth / containerHeight >= 4 / 3 ? 1 : TILE_ASPECT_PORTRAIT
let best: AdaptiveTileGrid | undefined
let bestScore: LayoutScore | undefined
for (let columnCount = 1; columnCount <= tileCount; columnCount++) {
const candidate = buildCandidate(
tileCount,
columnCount,
containerWidth,
containerHeight,
minAspect,
)
if (!candidate) {
continue
}
const {tileWidth, tileHeight, aspectRatio} = candidate.rows[0]
const verticalOverflow = Math.max(0, candidate.totalHeight - containerHeight)
const horizontalOverflow = Math.max(0, candidate.totalWidth - containerWidth)
const whitespace =
Math.max(0, containerWidth - candidate.totalWidth) +
Math.max(0, containerHeight - candidate.totalHeight)
const totalArea = tileWidth * tileHeight * tileCount
const overflowPenaltyMultiplier = tileHeight >= MIN_TILE_HEIGHT ? 1_000_000 : 1
const verticalOverflowPenalty = verticalOverflow * overflowPenaltyMultiplier
const horizontalOverflowPenalty = horizontalOverflow * overflowPenaltyMultiplier
const minTileHeightPenalty =
tileHeight < MIN_TILE_HEIGHT ? (MIN_TILE_HEIGHT - tileHeight) * 1000 : 0
const score: LayoutScore = {
verticalOverflowPenalty,
horizontalOverflowPenalty,
minTileHeightPenalty,
negativeArea: -totalArea,
whitespace,
aspectDeviation: Math.abs(aspectRatio - TILE_ASPECT_LANDSCAPE),
}
if (!bestScore || compareScores(score, bestScore) < 0) {
bestScore = score
best = candidate
}
}
return best
}
export type CallSession = {
url: string
h: string
@ -231,6 +56,11 @@ export const isCallActive = derived(
$state => $state === CallState.Joining || $state === CallState.Connected,
)
export const deriveRoomCallState = (url: string, h?: string) =>
derived([callState, callTargetRoom], ([$callState, $targetRoom]) =>
$targetRoom?.url === url && $targetRoom.h === h ? $callState : CallState.Disconnected,
)
export const deriveIsCallActiveElsewhere = (url: string | undefined, h: string | undefined) =>
derived(
[isCallActive, callTargetRoom],

View file

@ -13,11 +13,10 @@ import {
import {App} from "@capacitor/app"
import type {PluginListenerHandle} from "@capacitor/core"
import {get} from "svelte/store"
import {not, nthEq, reject} from "@welshman/lib"
import {makeHttpAuth, makeHttpAuthHeader} from "@welshman/util"
import {not, nthEq, reject, spec} from "@welshman/lib"
import {makeRoomKey} from "@welshman/app"
import {user} from "@app/core"
import {getLivekitEndpoint} from "@app/relays"
import {getHttpAuthHeader} from "@app/signer"
import {pushToast} from "@app/toast"
import {
CallState,
@ -32,6 +31,7 @@ import {
speakingParticipants,
triggerVideoTrackRevision,
videoPrimaryTileKey,
type CallSession,
type ParticipantMediaState,
} from "@app/call"
@ -443,12 +443,8 @@ const fetchLivekitToken = async (
throw new DOMException("Aborted", "AbortError")
}
const template = await makeHttpAuth(endpoint, "GET")
const signedEvent = await user.get().signer.sign(template)
const authHeader = makeHttpAuthHeader(signedEvent)
const response = await fetch(endpoint, {
headers: {Authorization: authHeader},
headers: {Authorization: await getHttpAuthHeader(endpoint)},
signal,
})
@ -667,3 +663,86 @@ const onLocalTrackUnpublished = (
const abortJoinVoiceRoom = () => {
joinAbortController?.abort()
}
export type VideoTile = {
liveKitIdentity: string
isLocal: boolean
trackSid: string
track: Track | undefined
source: Track.Source.Camera | Track.Source.ScreenShare
}
// LiveKit can change trackSid after publish, so a tile is keyed by identity and source.
export const getVideoTileKey = (tile: VideoTile) => `${tile.liveKitIdentity}\x1f${tile.source}`
export const getVideoTiles = ({livekit, cameraOn, screenShareOn}: CallSession) => {
const videoTiles: VideoTile[] = []
const user = livekit.localParticipant
if (cameraOn) {
const localPub = user.getTrackPublication(Track.Source.Camera)
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: localPub?.trackSid ?? "local-camera",
track: localPub?.track,
source: Track.Source.Camera,
})
}
if (screenShareOn) {
const localPub = user.getTrackPublication(Track.Source.ScreenShare)
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: localPub?.trackSid ?? "local-screen",
track: localPub?.track,
source: Track.Source.ScreenShare,
})
}
for (const rp of livekit.remoteParticipants.values()) {
const camPub = rp.getTrackPublication(Track.Source.Camera)
// Camera off mutes the publication rather than unsubscribing; still render avatar.
if (camPub?.isSubscribed && camPub.track && !camPub.isMuted) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: camPub.trackSid,
track: camPub.track,
source: Track.Source.Camera,
})
}
const screenPub = rp.getTrackPublication(Track.Source.ScreenShare)
if (screenPub?.isSubscribed && screenPub.track && !screenPub.isMuted) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: screenPub.trackSid,
track: screenPub.track,
source: Track.Source.ScreenShare,
})
}
if (!videoTiles.some(spec({liveKitIdentity: rp.identity}))) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: `avatar-${rp.identity}`,
track: undefined,
source: Track.Source.Camera,
})
}
}
if (!videoTiles.some(spec({liveKitIdentity: user.identity}))) {
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: "local-avatar",
track: undefined,
source: Track.Source.Camera,
})
}
return videoTiles
}

View file

@ -1,5 +1,6 @@
import {derived} from "svelte/store"
import type {Readable} from "svelte/store"
import {simpleCache} from "@welshman/lib"
import {
Address,
COMMAND,
@ -76,16 +77,8 @@ export const getCommandsForTarget = (target: CommandScopeTarget) =>
.filter(command => command.matches(target))
// One store per url, and subscribing to it is what pulls the definitions.
const commandsByUrl = new Map<string, Readable<CommandReader[]>>()
const deriveCommandsForUrl = (url: string): Readable<CommandReader[]> => {
const cached = commandsByUrl.get(url)
if (cached) {
return cached
}
const store = fromApp($app => {
const deriveCommandsForUrl = simpleCache(([url]: [string]) =>
fromApp($app => {
const plugin = $app.use(Commands)
if (url) {
@ -93,12 +86,8 @@ const deriveCommandsForUrl = (url: string): Readable<CommandReader[]> => {
}
return plugin.forUrl(url).$
})
commandsByUrl.set(url, store)
return store
}
}),
)
// Content outside a space has no url, and so no commands.
export const deriveValidCommands = (target: CommandScopeTarget): Readable<CommandReader[]> =>

View file

@ -26,9 +26,8 @@
videoCallLayout,
CallState,
currentCallSession,
callTargetRoom,
callMicMuted,
callState,
deriveRoomCallState,
} from "@app/call"
import {
cancelJoinVoiceRoom,
@ -49,12 +48,9 @@
const room = $derived($roomStore)
const isVoiceRoom = $derived(room !== undefined && getRoomType(room) === RoomType.Voice)
const isTargetingThisRoom = $derived(
$callTargetRoom !== undefined && $callTargetRoom.url === url && $callTargetRoom.h === h,
)
const joiningHere = $derived($callState === CallState.Joining && isTargetingThisRoom)
const connectedHere = $derived($callState === CallState.Connected && isTargetingThisRoom)
const roomCallState = $derived(deriveRoomCallState(url, h))
const joiningHere = $derived($roomCallState === CallState.Joining)
const connectedHere = $derived($roomCallState === CallState.Connected)
// callTargetRoom isn't cleared on cancel or leave, so it can still point at this room.
const showJoin = $derived(isVoiceRoom && !joiningHere && !connectedHere)

View file

@ -6,7 +6,7 @@
import RelayIcon from "@app/components/RelayIcon.svelte"
import RelayName from "@app/components/RelayName.svelte"
import {displayContentCount} from "@app/content"
import {inboxSpaceContent} from "@app/inbox"
import {inboxSpaceContent} from "@app/notifications"
import {makeSpacePath} from "@app/routes"
</script>

View file

@ -7,9 +7,8 @@
import Button from "@lib/components/Button.svelte"
import HomeSection from "@app/components/HomeSection.svelte"
import HomeInboxItem from "@app/components/HomeInboxItem.svelte"
import {inboxConversations} from "@app/inbox"
import {navigate} from "@app/modal"
import {allNotifications, setChecked} from "@app/notifications"
import {allNotifications, inboxConversations, setChecked} from "@app/notifications"
const conversations = $derived($inboxConversations.slice(0, 8))

View file

@ -2,6 +2,7 @@
import cx from "classnames"
import Microphone from "@assets/icons/microphone.svg?dataurl"
import Icon from "@lib/components/Icon.svelte"
import {watchAudioLevel} from "@lib/audio"
type Props = {
track: MediaStreamTrack | undefined
@ -11,57 +12,34 @@
const {track, error, offMessage = "Microphone is off"}: Props = $props()
const NOISE_FLOOR = 0.03
let level = $state(0)
$effect(() => {
const t = track
level = 0
if (!t || error) {
return
}
if (t && !error) {
// Reading the `$state` this effect writes would make it depend on its own output.
let smoothed = 0
let frame: number | undefined
const context = new AudioContext()
const analyser = context.createAnalyser()
analyser.fftSize = 512
context.createMediaStreamSource(new MediaStream([t])).connect(analyser)
const data = new Uint8Array(analyser.frequencyBinCount)
return watchAudioLevel(new MediaStream([t]), rms => {
if (t.enabled && t.readyState === "live") {
const gated = rms < NOISE_FLOOR ? 0 : (rms - NOISE_FLOOR) / (1 - NOISE_FLOOR)
// Reading the `$state` this effect writes would make it depend on its own output.
let smoothed = 0
smoothed += (Math.min(1, gated * 4) - smoothed) * 0.5
const tick = () => {
if (!t.enabled || t.readyState !== "live") {
smoothed = 0
level = 0
frame = requestAnimationFrame(tick)
return
}
if (smoothed < 0.005) {
smoothed = 0
}
} else {
smoothed = 0
}
analyser.getByteTimeDomainData(data)
let sumSquares = 0
for (const v of data) {
const normalized = (v - 128) / 128
sumSquares += normalized * normalized
}
const rms = Math.sqrt(sumSquares / data.length)
const NOISE_FLOOR = 0.03
const gated = rms < NOISE_FLOOR ? 0 : (rms - NOISE_FLOOR) / (1 - NOISE_FLOOR)
smoothed += (Math.min(1, gated * 4) - smoothed) * 0.5
if (smoothed < 0.005) {
smoothed = 0
}
level = smoothed
frame = requestAnimationFrame(tick)
}
tick()
return () => {
if (frame !== undefined) {
cancelAnimationFrame(frame)
}
context.close().catch(() => {})
level = smoothed
})
}
})
</script>

View file

@ -19,7 +19,13 @@
import ThunkToast from "@app/components/ThunkToast.svelte"
import CallControlBar from "@app/components/CallControlBar.svelte"
import {deletes, relays, rooms, thunks, user, writer} from "@app/core"
import {CallState, callTargetRoom, callState, VideoCallLayout, videoCallLayout} from "@app/call"
import {
CallState,
callState,
deriveRoomCallState,
VideoCallLayout,
videoCallLayout,
} from "@app/call"
import {
RoomType,
deriveUserRoomMembershipStatus,
@ -42,11 +48,9 @@
const room = h ? $rooms.forRoom(url, h) : readable(undefined)
const isVoiceRoom = $derived($room && getRoomType($room) === RoomType.Voice)
const isCallTargetingThisRoom = $derived($callTargetRoom?.url === url && $callTargetRoom?.h === h)
const roomCallState = deriveRoomCallState(url, h)
const voiceConnectedHere = $derived(
isVoiceRoom && $callState === CallState.Connected && isCallTargetingThisRoom,
)
const voiceConnectedHere = $derived(isVoiceRoom && $roomCallState === CallState.Connected)
// voiceConnectedHere can't stay true for a voice room the call isn't targeting.
const callChatOpen = $derived(voiceConnectedHere && $videoCallLayout === VideoCallLayout.Split)

View file

@ -1,6 +1,6 @@
<script lang="ts">
import StatusIndicator from "@lib/components/StatusIndicator.svelte"
import {deriveSocketStatus} from "@app/relays"
import {RelayConnection, deriveSocketStatus} from "@app/relays"
type Props = {
url: string
@ -10,4 +10,18 @@
const status = deriveSocketStatus(url)
</script>
<StatusIndicator class="bg-{$status.theme}">{$status.title}</StatusIndicator>
{#if $status === RelayConnection.Connecting}
<StatusIndicator class="bg-warning">Connecting</StatusIndicator>
{:else if $status === RelayConnection.Disconnected}
<StatusIndicator class="bg-gray-500">Not Connected</StatusIndicator>
{:else if $status === RelayConnection.ConnectFailed}
<StatusIndicator class="bg-error">Failed to Connect</StatusIndicator>
{:else if $status === RelayConnection.Authenticating}
<StatusIndicator class="bg-warning">Authenticating</StatusIndicator>
{:else if $status === RelayConnection.AuthFailed}
<StatusIndicator class="bg-error">Failed to Authenticate</StatusIndicator>
{:else if $status === RelayConnection.AccessDenied}
<StatusIndicator class="bg-error">Access Denied</StatusIndicator>
{:else}
<StatusIndicator class="bg-success">Connected</StatusIndicator>
{/if}

View file

@ -1,7 +1,7 @@
<script lang="ts">
import {onDestroy} from "svelte"
import SocketStatusIndicator from "@app/components/SocketStatusIndicator.svelte"
import {deriveSocketStatus} from "@app/relays"
import {RelayConnection, deriveSocketStatus} from "@app/relays"
import {pushToast, popToast} from "@app/toast"
type Props = {
@ -26,7 +26,7 @@
}
$effect(() => {
if ($status.theme !== "warning") {
if ($status !== RelayConnection.Connecting && $status !== RelayConnection.Authenticating) {
hide()
return
}

View file

@ -1,10 +1,12 @@
<script lang="ts">
import {removeUndefined, spec} from "@welshman/lib"
import {removeUndefined} from "@welshman/lib"
import cx from "classnames"
import {Track} from "livekit-client"
import Pin from "@assets/icons/pin.svg?dataurl"
import Button from "@lib/components/Button.svelte"
import Icon from "@lib/components/Icon.svelte"
import {computeAdaptiveGrid} from "@lib/tileGrid"
import type {AdaptiveTileGrid} from "@lib/tileGrid"
import ProfileCircle from "@app/components/ProfileCircle.svelte"
import VideoCallTile from "@app/components/VideoCallTile.svelte"
import VoiceParticipantMediaBadges from "@app/components/VoiceParticipantMediaBadges.svelte"
@ -17,9 +19,9 @@
participantMediaState,
pubkeyFromLiveKitIdentity,
videoTrackRevision,
computeAdaptiveGrid,
type AdaptiveTileGrid,
} from "@app/call"
import {getVideoTileKey, getVideoTiles} from "@app/callEngine"
import type {VideoTile} from "@app/callEngine"
import {deriveDisplaysByPubkey} from "@app/social"
type Props = {
@ -28,108 +30,26 @@
class?: string
}
type VideoTileData = {
liveKitIdentity: string
isLocal: boolean
trackSid: string
track: Track | undefined
source: Track.Source.Camera | Track.Source.ScreenShare
}
type TileLayoutVariant = "spotlight" | "default" | "strip"
const {url, h, class: className}: Props = $props()
const videoTiles = $derived.by(() => {
const session = $currentCallSession
// LiveKit mutates remoteParticipants and tracks in place, so these stores are what change.
void $participantMediaState
void $videoTrackRevision
if (!session || $callTargetRoom?.url !== url || $callTargetRoom?.h !== h) {
return []
}
const livekit = session.livekit
const videoTiles: VideoTileData[] = []
const user = livekit.localParticipant
if (session.cameraOn) {
const localPub = user.getTrackPublication(Track.Source.Camera)
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: localPub?.trackSid ?? "local-camera",
track: localPub?.track,
source: Track.Source.Camera,
})
}
if (session.screenShareOn) {
const localPub = user.getTrackPublication(Track.Source.ScreenShare)
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: localPub?.trackSid ?? "local-screen",
track: localPub?.track,
source: Track.Source.ScreenShare,
})
}
for (const rp of livekit.remoteParticipants.values()) {
const camPub = rp.getTrackPublication(Track.Source.Camera)
// Camera off mutes the publication rather than unsubscribing; still render avatar.
if (camPub?.isSubscribed && camPub.track && !camPub.isMuted) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: camPub.trackSid,
track: camPub.track,
source: Track.Source.Camera,
})
}
const screenPub = rp.getTrackPublication(Track.Source.ScreenShare)
if (screenPub?.isSubscribed && screenPub.track && !screenPub.isMuted) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: screenPub.trackSid,
track: screenPub.track,
source: Track.Source.ScreenShare,
})
}
if (!videoTiles.some(spec({liveKitIdentity: rp.identity}))) {
videoTiles.push({
liveKitIdentity: rp.identity,
isLocal: false,
trackSid: `avatar-${rp.identity}`,
track: undefined,
source: Track.Source.Camera,
})
}
}
if (!videoTiles.some(spec({liveKitIdentity: user.identity}))) {
videoTiles.push({
liveKitIdentity: user.identity,
isLocal: true,
trackSid: "local-avatar",
track: undefined,
source: Track.Source.Camera,
})
}
return videoTiles
return $currentCallSession && $callTargetRoom?.url === url && $callTargetRoom?.h === h
? getVideoTiles($currentCallSession)
: []
})
/** LiveKit identity + source only — LiveKit can change trackSid after publish, which broke spotlight + stale-key effect. */
const tileKey = (t: VideoTileData) => `${t.liveKitIdentity}\x1f${t.source}`
const primaryTile = $derived.by(() => {
const k = $videoPrimaryTileKey
if (k === undefined) {
return undefined
}
return videoTiles.find(t => tileKey(t) === k)
return videoTiles.find(t => getVideoTileKey(t) === k)
})
const secondaryTiles = $derived.by(() => {
@ -137,8 +57,8 @@
if (p === undefined) {
return videoTiles
}
const pk = tileKey(p)
return videoTiles.filter(t => tileKey(t) !== pk)
const pk = getVideoTileKey(p)
return videoTiles.filter(t => getVideoTileKey(t) !== pk)
})
let gridWidth = $state(0)
@ -156,7 +76,7 @@
if (k === undefined) {
return
}
if (!videoTiles.some(t => tileKey(t) === k)) {
if (!videoTiles.some(t => getVideoTileKey(t) === k)) {
videoPrimaryTileKey.set(undefined)
}
})
@ -168,7 +88,7 @@
),
)
const labelFor = (liveKitIdentity: string, source: VideoTileData["source"]) => {
const labelFor = (liveKitIdentity: string, source: VideoTile["source"]) => {
const pk = pubkeyFromLiveKitIdentity(liveKitIdentity)
const name = (pk && $displays.get(pk)) || "Unknown"
@ -182,7 +102,7 @@
}
</script>
{#snippet videoTile(tile: VideoTileData, layout: TileLayoutVariant)}
{#snippet videoTile(tile: VideoTile, layout: TileLayoutVariant)}
{@const media = $mediaStateByIdentity(tile.liveKitIdentity)}
{@const label = labelFor(tile.liveKitIdentity, tile.source)}
<div
@ -219,7 +139,7 @@
{label}{tile.isLocal ? " (you)" : ""}
</span>
{#if videoTiles.length > 1}
{@const pinned = $videoPrimaryTileKey === tileKey(tile)}
{@const pinned = $videoPrimaryTileKey === getVideoTileKey(tile)}
<Button
data-tip={pinned ? "Exit spotlight" : "Spotlight"}
aria-label={pinned ? `Exit spotlight for ${label}` : `Spotlight ${label}`}
@ -229,7 +149,7 @@
"absolute right-1 top-1 z-20",
!pinned && "bg-surface",
)}
onclick={spotlightHandlerFor(tileKey(tile))}>
onclick={spotlightHandlerFor(getVideoTileKey(tile))}>
<Icon icon={Pin} size={3} />
</Button>
{/if}
@ -244,7 +164,7 @@
{#if secondaryTiles.length > 0}
<div
class="flex max-h-40 shrink-0 flex-row gap-2 overflow-x-auto overflow-y-hidden py-0.5">
{#each secondaryTiles as tile (tileKey(tile))}
{#each secondaryTiles as tile (getVideoTileKey(tile))}
{@render videoTile(tile, "strip")}
{/each}
</div>
@ -264,7 +184,7 @@
<div
class="flex flex-nowrap justify-center gap-2"
style={`max-width: ${row.rowWidth}px`}>
{#each videoTiles.slice(offset, offset + row.columnCount) as tile (tileKey(tile))}
{#each videoTiles.slice(offset, offset + row.columnCount) as tile (getVideoTileKey(tile))}
<div
class="overflow-hidden rounded-2xl shrink-0"
style={`width: ${row.tileWidth}px; height: ${row.tileHeight}px`}>

View file

@ -9,6 +9,7 @@
import ModalHeader from "@lib/components/ModalHeader.svelte"
import ModalSubtitle from "@lib/components/ModalSubtitle.svelte"
import ModalTitle from "@lib/components/ModalTitle.svelte"
import {listMediaDevices} from "@lib/audio"
import MicLevelMeter from "@app/components/MicLevelMeter.svelte"
import {currentCallSession, DeviceKind, type CallSession} from "@app/call"
import {switchCallActiveDevice} from "@app/callEngine"
@ -35,19 +36,11 @@
?.mediaStreamTrack
const loadDevices = async () => {
if (!navigator.mediaDevices?.enumerateDevices) {
return
}
try {
const devices = await navigator.mediaDevices.enumerateDevices()
audioInputs = devices.filter(spec({kind: "audioinput"}))
audioOutputs = devices.filter(spec({kind: "audiooutput"}))
videoInputs = devices.filter(spec({kind: "videoinput"}))
} catch {
audioInputs = []
audioOutputs = []
videoInputs = []
}
const devices = await listMediaDevices()
audioInputs = devices.filter(spec({kind: "audioinput"}))
audioOutputs = devices.filter(spec({kind: "audiooutput"}))
videoInputs = devices.filter(spec({kind: "videoinput"}))
}
$effect(() => {

View file

@ -1,6 +1,5 @@
<script lang="ts">
import cx from "classnames"
import {makeRoomKey} from "@welshman/app"
import SecondaryNavItem from "@lib/components/SecondaryNavItem.svelte"
import Spinner from "@lib/components/Spinner.svelte"
import ProfileCircle from "@app/components/ProfileCircle.svelte"
@ -14,12 +13,11 @@
import VoiceParticipantMediaBadges from "@app/components/VoiceParticipantMediaBadges.svelte"
import {
CallState,
callTargetRoom,
isParticipantSpeaking,
mediaStateByIdentity,
participantKey,
callState,
deriveCallParticipants,
deriveRoomCallState,
loadCallParticipants,
type CallParticipant,
} from "@app/call"
@ -39,12 +37,9 @@
const participants = deriveCallParticipants(url, h)
const participantPubkeys = $derived($participants.flatMap(p => (p.pubkey ? [p.pubkey] : [])))
const isActive = $derived(
$callState === CallState.Connected && $callTargetRoom?.id === makeRoomKey(url, h),
)
const isJoining = $derived(
$callState === CallState.Joining && $callTargetRoom?.id === makeRoomKey(url, h),
)
const roomCallState = deriveRoomCallState(url, h)
const isActive = $derived($roomCallState === CallState.Connected)
const isJoining = $derived($roomCallState === CallState.Joining)
const handleClick = async (e: MouseEvent) => {
if (isActive) {

View file

@ -12,6 +12,7 @@
import ModalHeader from "@lib/components/ModalHeader.svelte"
import ModalSubtitle from "@lib/components/ModalSubtitle.svelte"
import ModalTitle from "@lib/components/ModalTitle.svelte"
import {listMediaDevices} from "@lib/audio"
import MicLevelMeter from "@app/components/MicLevelMeter.svelte"
import ProfileCircles from "@app/components/ProfileCircles.svelte"
import {displayRoom} from "@app/rooms"
@ -37,15 +38,7 @@
let micError = $state(false)
const loadDevices = async () => {
if (!navigator.mediaDevices?.enumerateDevices) {
return
}
try {
const devices = await navigator.mediaDevices.enumerateDevices()
audioInputs = devices.filter(spec({kind: "audioinput"}))
} catch {
audioInputs = []
}
audioInputs = (await listMediaDevices()).filter(spec({kind: "audioinput"}))
}
$effect(() => {

View file

@ -1,4 +1,5 @@
import {getSetting} from "@app/settings"
import {watchAudioLevel} from "@lib/audio"
import {postOpenRouter} from "@app/openrouter"
const TRANSCRIPTION_MODEL = "openai/whisper-large-v3-turbo"
@ -17,17 +18,8 @@ const transcribe = async (audio: File) => {
body.append("model", TRANSCRIPTION_MODEL)
body.append("file", audio, audio.name)
const response = await fetch("https://openrouter.ai/api/v1/audio/transcriptions", {
method: "POST",
headers: {Authorization: `Bearer ${getSetting("openrouter_key")}`},
body,
})
const {text, error}: {text?: string; error?: {message?: string}} = await response.json()
if (!response.ok) {
throw new Error(error?.message || `OpenRouter returned a ${response.status}.`)
}
const response = await postOpenRouter("audio/transcriptions", body)
const {text}: {text?: string} = await response.json()
return text?.trim() ?? ""
}
@ -70,39 +62,17 @@ export const startDictation = async (key: string, onLevel: (level: number) => vo
recorder.addEventListener("dataavailable", event => chunks.push(event.data))
recorder.start()
const context = new AudioContext()
const analyser = context.createAnalyser()
analyser.fftSize = 512
context.createMediaStreamSource(stream).connect(analyser)
const samples = new Uint8Array(analyser.frequencyBinCount)
let level = 0
const measure = () => {
analyser.getByteTimeDomainData(samples)
let sumSquares = 0
for (const sample of samples) {
sumSquares += ((sample - 128) / 128) ** 2
}
level = Math.max(Math.sqrt(sumSquares / samples.length), level * 0.92)
const stopMeter = watchAudioLevel(stream, rms => {
level = Math.max(rms, level * 0.92)
onLevel(level)
frame = requestAnimationFrame(measure)
}
let frame = requestAnimationFrame(measure)
})
const audio = new Promise<File>(resolve => {
recorder.addEventListener("stop", () => {
cancelAnimationFrame(frame)
context.close()
stopMeter()
for (const track of stream.getTracks()) {
track.stop()

View file

@ -3,12 +3,13 @@ import type {Readable} from "svelte/store"
import {Capacitor} from "@capacitor/core"
import type {Maybe} from "@welshman/lib"
import {sortBy, spec, int, MINUTE, ms} from "@welshman/lib"
import {makeHttpAuth, makeHttpAuthHeader, normalizeRelayUrl} from "@welshman/util"
import {normalizeRelayUrl} from "@welshman/util"
import {User} from "@welshman/app"
import type {IApp} from "@welshman/app"
import {roomLists, usePlugin, user} from "@app/core"
import {HOSTING_BACKEND_URL, HOSTING_RELAY_DOMAIN, PLATFORM_URL} from "@app/env"
import {goToMovedSpace} from "@app/routes"
import {getHttpAuthHeader} from "@app/signer"
// Apple doesn't allow selling hosting outside their payment system.
export const HOSTING_ENABLED = Capacitor.getPlatform() !== "ios"
@ -180,32 +181,6 @@ export const hostingErrorMessage = (e: unknown, fallback: string) => {
// Requests
const AUTH_TTL = ms(int(10, MINUTE))
// The backend checks the signature and `u` tag but not freshness, so one token per pubkey is reusable.
let cachedAuth: Maybe<{pubkey: string; expiresAt: number; header: Promise<string>}>
const getAuthHeader = (): Promise<string> => {
const {pubkey, signer} = user.get()
if (cachedAuth?.pubkey !== pubkey || cachedAuth.expiresAt < Date.now()) {
const header = makeHttpAuth(HOSTING_BACKEND_URL, "GET")
.then(template => signer.sign(template))
.then(makeHttpAuthHeader)
// A declined signature shouldn't be cached, or the user can't retry.
header.catch(() => {
if (cachedAuth?.header === header) {
cachedAuth = undefined
}
})
cachedAuth = {pubkey, expiresAt: Date.now() + AUTH_TTL, header}
}
return cachedAuth.header
}
type HostingResponse<T> = {data: T; error?: string}
const hostingRequest = async (method: string, path: string, init: RequestInit = {}) =>
@ -214,7 +189,8 @@ const hostingRequest = async (method: string, path: string, init: RequestInit =
method,
headers: {
Accept: "application/json",
Authorization: await getAuthHeader(),
// The backend checks the signature and `u` tag but not freshness, so a token is reusable.
Authorization: await getHttpAuthHeader(HOSTING_BACKEND_URL, int(10, MINUTE)),
...init.headers,
},
})

View file

@ -1,41 +0,0 @@
import {derived} from "svelte/store"
import {sortBy} from "@welshman/lib"
import {allNotifications, latestActivityByPath} from "@app/notifications"
// The inbox shows what the badges show, so reading a conversation takes it out of the list.
export const inboxConversations = derived(
[latestActivityByPath, allNotifications],
([$latestActivityByPath, $allNotifications]) =>
sortBy(
activity => -activity.event.created_at,
[...$latestActivityByPath.values()].filter(
activity => !activity.contentKind && $allNotifications.has(activity.path),
),
),
)
export type SpaceContent = {
url: string
timestamp: number
countsByKind: Map<number, number>
}
// Counted only where unread, since a total would be the same number every day.
export const inboxSpaceContent = derived(
[latestActivityByPath, allNotifications],
([$latestActivityByPath, $allNotifications]) => {
const byUrl = new Map<string, SpaceContent>()
for (const {path, url, contentKind, event} of $latestActivityByPath.values()) {
if (url && contentKind && $allNotifications.has(path)) {
const content = byUrl.get(url) ?? {url, timestamp: 0, countsByKind: new Map()}
content.timestamp = Math.max(content.timestamp, event.created_at)
content.countsByKind.set(contentKind, (content.countsByKind.get(contentKind) ?? 0) + 1)
byUrl.set(url, content)
}
}
return sortBy(content => -content.timestamp, [...byUrl.values()])
},
)

View file

@ -1,18 +1,22 @@
import {derived, get, writable} from "svelte/store"
import {Badge} from "@capawesome/capacitor-badge"
import {page} from "$app/stores"
import {assoc, first, identity, groupBy, now, remove, throttle, parseJson, gt} from "@welshman/lib"
import type {SignedEvent, TrustedEvent} from "@welshman/util"
import {
getIdOrAddress,
sortEventsDesc,
tagSpec,
tagValue,
COMMENT,
MESSAGE,
makeHttpAuth,
makeHttpAuthHeader,
} from "@welshman/util"
assoc,
first,
identity,
groupBy,
now,
remove,
sortBy,
throttle,
parseJson,
gt,
int,
HOUR,
} from "@welshman/lib"
import type {TrustedEvent} from "@welshman/util"
import {getIdOrAddress, sortEventsDesc, tagSpec, tagValue, COMMENT, MESSAGE} from "@welshman/util"
import {Comment} from "@welshman/domain"
import {throttled} from "@welshman/store"
import {Events, Relays} from "@welshman/app"
@ -24,6 +28,7 @@ import {Chats} from "@app/chats"
import {dufflepud, DUFFLEPUD_URL} from "@app/env"
import {isGroupSpace} from "@app/relays"
import {activeSpaceUrls} from "@app/rooms"
import {getHttpAuthHeader} from "@app/signer"
import {kv, syncedWithGetter} from "@app/storage"
import {notificationSettings} from "@app/push/adapters/common"
@ -65,14 +70,13 @@ export const syncChecked = () => {
// Set checked when we visit a given page - but delay it a tad
setTimeout(() => {
const defer = get(deferredRoomPath)
const $deferredRoomPath = get(deferredRoomPath)
checked.update($checked => {
for (const path of paths) {
if (defer && path === defer) {
continue
if (path !== $deferredRoomPath) {
$checked[path] = now()
}
$checked[path] = now()
}
return $checked
@ -84,31 +88,10 @@ export const syncChecked = () => {
}
const CHECKED_KV_KEY = "checked"
const NIP98_MAX_AGE = 23 * 60 * 60
let nip98Auth: SignedEvent | undefined
const nip98Header = async () => {
const $signer = app.get().user?.signer
if (!$signer) {
return undefined
}
if (!nip98Auth || now() - nip98Auth.created_at > NIP98_MAX_AGE) {
nip98Auth = await $signer.sign(await makeHttpAuth(DUFFLEPUD_URL, "GET"))
}
return makeHttpAuthHeader(nip98Auth)
}
const getAuthHeader = () => getHttpAuthHeader(DUFFLEPUD_URL, int(23, HOUR))
const pullCheckedRemote = async () => {
const authorization = await nip98Header()
if (!authorization) {
return
}
const authorization = await getAuthHeader()
const res = await fetch(dufflepud(`kv/${CHECKED_KV_KEY}`), {headers: {authorization}})
if (!res.ok) {
@ -133,11 +116,7 @@ const pullCheckedRemote = async () => {
}
const pushCheckedRemote = throttle(3000, async () => {
const authorization = await nip98Header()
if (!authorization) {
return
}
const authorization = await getAuthHeader()
try {
await fetch(dufflepud(`kv/${CHECKED_KV_KEY}`), {
@ -155,7 +134,6 @@ export const syncCheckedRemote = () => {
const unsubscribeUser = app.subscribe($app => {
ready = false
nip98Auth = undefined
if ($app.user) {
pullCheckedRemote().then(() => {
@ -331,15 +309,9 @@ export const notifications = derived(
[page, allNotifications, deferredRoomPath],
([$page, $allNotifications, $deferredRoomPath]) =>
new Set(
[...$allNotifications].filter(p => {
if (!$page.url.pathname.startsWith(p)) {
return true
}
if ($deferredRoomPath && p === $deferredRoomPath) {
return true
}
return false
}),
[...$allNotifications].filter(
p => !$page.url.pathname.startsWith(p) || p === $deferredRoomPath,
),
),
)
@ -381,3 +353,43 @@ export const clearBadges = async () => {
// pass - firefox doesn't support this
}
}
// Inbox
// The inbox shows what the badges show, so reading a conversation takes it out of the list.
export const inboxConversations = derived(
[latestActivityByPath, allNotifications],
([$latestActivityByPath, $allNotifications]) =>
sortBy(
activity => -activity.event.created_at,
[...$latestActivityByPath.values()].filter(
activity => !activity.contentKind && $allNotifications.has(activity.path),
),
),
)
export type SpaceContent = {
url: string
timestamp: number
countsByKind: Map<number, number>
}
// Counted only where unread, since a total would be the same number every day.
export const inboxSpaceContent = derived(
[latestActivityByPath, allNotifications],
([$latestActivityByPath, $allNotifications]) => {
const byUrl = new Map<string, SpaceContent>()
for (const {path, url, contentKind, event} of $latestActivityByPath.values()) {
if (url && contentKind && $allNotifications.has(path)) {
const content = byUrl.get(url) ?? {url, timestamp: 0, countsByKind: new Map()}
content.timestamp = Math.max(content.timestamp, event.created_at)
content.countsByKind.set(contentKind, (content.countsByKind.get(contentKind) ?? 0) + 1)
byUrl.set(url, content)
}
}
return sortBy(content => -content.timestamp, [...byUrl.values()])
},
)

25
src/app/openrouter.ts Normal file
View file

@ -0,0 +1,25 @@
import type {Maybe} from "@welshman/lib"
import {getSetting} from "@app/settings"
// A successful response can be audio rather than json, so only a failure is read as json.
export const postOpenRouter = async (
path: string,
body: BodyInit,
headers: Record<string, string> = {},
) => {
const response = await fetch(`https://openrouter.ai/api/v1/${path}`, {
method: "POST",
headers: {Authorization: `Bearer ${getSetting("openrouter_key")}`, ...headers},
body,
})
if (!response.ok) {
const payload: Maybe<{error?: {message?: string}}> = await response
.json()
.catch(() => undefined)
throw new Error(payload?.error?.message || `OpenRouter returned a ${response.status}.`)
}
return response
}

View file

@ -4,15 +4,18 @@ import {assoc, hash, maybe, ms, poll} from "@welshman/lib"
import type {Filter} from "@welshman/util"
import {Address, DELETE, makeEvent} from "@welshman/util"
import {Relays, User} from "@welshman/app"
import {buildUrl} from "@lib/util"
import {app, messagingRelayLists, roomLists, thunks} from "@app/core"
import {device} from "@app/device"
import {PUSH_BRIDGE, PUSH_SERVER} from "@app/env"
import {PUSH_BRIDGE} from "@app/env"
import {pushState} from "@app/push/adapters/common"
import type {IPushAdapter} from "@app/push/adapters/common"
import {onPushNotificationAction, syncRelaySubscriptions} from "@app/push/adapters/common"
import {createPushSubscription, deletePushSubscription} from "@app/push/server"
import {syncEventReminders} from "@app/reminders"
// NIP-9a: asks a relay to post matching events to the push server's callback
const PUSH_SUBSCRIPTION = 30390
const requestPermissions = async () => {
let status = await PushNotifications.checkPermissions()
@ -71,16 +74,7 @@ export class CapacitorNotifications implements IPushAdapter {
if (!subscription) {
try {
const channel = Capacitor.getPlatform() === "ios" ? "apns" : "fcm"
const url = buildUrl(PUSH_SERVER, "subscription", channel)
const res = await fetch(url, {
signal,
method: "POST",
body: JSON.stringify({token}),
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
})
const res = await createPushSubscription(channel, token, signal)
if (!res.ok) {
console.warn(`Failed to register with push server (status ${res.status})`)
@ -131,7 +125,7 @@ export class CapacitorNotifications implements IPushAdapter {
const thunk = thunks.get().publish({
relays: [url],
event: makeEvent(30390, {
event: makeEvent(PUSH_SUBSCRIPTION, {
tags: [
["d", identifier],
["relay", relay],
@ -160,7 +154,11 @@ export class CapacitorNotifications implements IPushAdapter {
const relays = [url]
const identifier = this._getSubscriptionIdentifier(relay, key)
const address = new Address(30390, User.require(app.get()).pubkey, identifier).toString()
const address = new Address(
PUSH_SUBSCRIPTION,
User.require(app.get()).pubkey,
identifier,
).toString()
const event = makeEvent(DELETE, {tags: [["a", address]]})
const error = await thunks.get().publish({relays, event}).waitForError()
@ -204,9 +202,7 @@ export class CapacitorNotifications implements IPushAdapter {
const {subscription} = pushState.get()
if (subscription) {
const res = await fetch(buildUrl(PUSH_SERVER, "subscription", subscription.key), {
method: "delete",
})
const res = await deletePushSubscription(subscription.key)
if (!res.ok) {
console.warn("Failed to delete push subscription")

40
src/app/push/server.ts Normal file
View file

@ -0,0 +1,40 @@
import type {SignedEvent} from "@welshman/util"
import {buildUrl} from "@lib/util"
import {PUSH_SERVER} from "@app/env"
export const createPushSubscription = (channel: string, token: string, signal: AbortSignal) =>
fetch(buildUrl(PUSH_SERVER, "subscription", channel), {
signal,
method: "POST",
body: JSON.stringify({token}),
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
})
export const deletePushSubscription = (key: string) =>
fetch(buildUrl(PUSH_SERVER, "subscription", key), {method: "delete"})
export type Reminder = {
at: number
title: string
body: string
}
export type ReminderSchedule = {
id: string
relay: string
event?: SignedEvent
notifications: Reminder[]
}
export const putReminderSchedule = (key: string, schedule: ReminderSchedule) =>
fetch(buildUrl(PUSH_SERVER, "subscription", key, "schedule"), {
method: "PUT",
headers: {Accept: "application/json", "Content-Type": "application/json"},
body: JSON.stringify(schedule),
})
export const deleteReminderSchedule = (key: string) =>
fetch(buildUrl(PUSH_SERVER, "subscription", key, "schedule"), {method: "DELETE"})

View file

@ -47,51 +47,49 @@ export const deriveSocket = (url: string) => {
})
}
export enum RelayConnection {
Connecting = "connecting",
Disconnected = "disconnected",
ConnectFailed = "connect-failed",
Authenticating = "authenticating",
AuthFailed = "auth-failed",
AccessDenied = "access-denied",
Connected = "connected",
}
export const deriveSocketStatus = (url: string) =>
throttled(
800,
derived([deriveSocket(url), relaysMostlyRestricted], ([$socket, $relaysMostlyRestricted]) => {
if ($socket.status === SocketStatus.Opening) {
return {theme: "warning", title: "Connecting"}
return RelayConnection.Connecting
}
if ($socket.status === SocketStatus.Closing) {
return {theme: "gray-500", title: "Not Connected"}
}
if ($socket.status === SocketStatus.Closed) {
return {theme: "gray-500", title: "Not Connected"}
if ($socket.status === SocketStatus.Closing || $socket.status === SocketStatus.Closed) {
return RelayConnection.Disconnected
}
if ($socket.status === SocketStatus.Error) {
return {theme: "error", title: "Failed to Connect"}
return RelayConnection.ConnectFailed
}
if ($socket.auth.status === AuthStatus.Requested) {
return {theme: "warning", title: "Authenticating"}
}
if ($socket.auth.status === AuthStatus.PendingSignature) {
return {theme: "warning", title: "Authenticating"}
if (
$socket.auth.status === AuthStatus.Requested ||
$socket.auth.status === AuthStatus.PendingSignature ||
$socket.auth.status === AuthStatus.PendingResponse
) {
return RelayConnection.Authenticating
}
if ($socket.auth.status === AuthStatus.DeniedSignature) {
return {theme: "error", title: "Failed to Authenticate"}
return RelayConnection.AuthFailed
}
if ($socket.auth.status === AuthStatus.PendingResponse) {
return {theme: "warning", title: "Authenticating"}
if ($socket.auth.status === AuthStatus.Forbidden || $relaysMostlyRestricted[url]) {
return RelayConnection.AccessDenied
}
if ($socket.auth.status === AuthStatus.Forbidden) {
return {theme: "error", title: "Access Denied"}
}
if ($relaysMostlyRestricted[url]) {
return {theme: "error", title: "Access Denied"}
}
return {theme: "success", title: "Connected"}
return RelayConnection.Connected
}),
)

View file

@ -2,23 +2,11 @@ import {MINUTE, first, formatTimestampRelative, int, ms, now, uniq} from "@welsh
import {EVENT_RSVP, getIdFilters, isSignedEvent} from "@welshman/util"
import type {TrustedEvent} from "@welshman/util"
import {CalendarRsvp} from "@welshman/domain"
import {buildUrl} from "@lib/util"
import {getEventStart, getEventTitle, isAttending} from "@app/calendar"
import {app, network, reader, roomLists} from "@app/core"
import {PUSH_SERVER} from "@app/env"
import {notificationSettings, pushState} from "@app/push/adapters/common"
type Reminder = {
at: number
title: string
body: string
}
const getScheduleUrl = () => {
const {subscription} = pushState.get()
return subscription && buildUrl(PUSH_SERVER, "subscription", subscription.key, "schedule")
}
import {deleteReminderSchedule, putReminderSchedule} from "@app/push/server"
import type {Reminder} from "@app/push/server"
const getLeadTimes = () => {
const {events, reminder_minutes} = notificationSettings.get()
@ -36,18 +24,14 @@ const makeReminders = (event: TrustedEvent, start: number): Reminder[] =>
.filter(({at}) => at > now())
const putSchedule = async (relay: string, event: TrustedEvent, reminders: Reminder[]) => {
const url = getScheduleUrl()
const {subscription} = pushState.get()
if (url) {
const res = await fetch(url, {
method: "PUT",
headers: {Accept: "application/json", "Content-Type": "application/json"},
body: JSON.stringify({
id: event.id,
relay,
event: isSignedEvent(event) ? event : undefined,
notifications: reminders,
}),
if (subscription) {
const res = await putReminderSchedule(subscription.key, {
id: event.id,
relay,
event: isSignedEvent(event) ? event : undefined,
notifications: reminders,
})
if (!res.ok) {
@ -97,11 +81,11 @@ const loadAttendedEvents = async (pubkey: string) => {
// Every schedule the push server holds was built from these settings, so redo the lot.
export const syncEventReminders = async () => {
const url = getScheduleUrl()
const {subscription} = pushState.get()
const pubkey = app.get().user?.pubkey
if (url && pubkey) {
const res = await fetch(url, {method: "DELETE"})
if (subscription && pubkey) {
const res = await deleteReminderSchedule(subscription.key)
if (!res.ok) {
console.warn(`Failed to clear event reminders (status ${res.status})`)

View file

@ -1,6 +1,6 @@
import type {Writable} from "svelte/store"
import {derived, writable} from "svelte/store"
import {append, equals, remove, spec} from "@welshman/lib"
import {append, equals, remove, spec, toggle} from "@welshman/lib"
import {APP_DATA} from "@welshman/util"
import {withGetter} from "@welshman/store"
import {AppData} from "@welshman/domain"
@ -180,44 +180,24 @@ export const addTrustedRelay = (url: string) =>
export const removeTrustedRelay = (url: string) =>
publishSettings({trusted_relays: remove(url, getSetting("trusted_relays"))})
export const setSpaceNotifications = (url: string, notify: boolean) => {
const updateSpaceAlert = (
url: string,
update: (alert: SpaceNotificationSettings) => SpaceNotificationSettings,
) => {
const alerts = getSetting("alerts")
const existing = alerts.find(spec({url}))
if (existing) {
return publishSettings({
alerts: alerts.map(s => (s.url === url ? {...s, notify, exceptions: []} : s)),
})
if (alerts.some(spec({url}))) {
return publishSettings({alerts: alerts.map(s => (s.url === url ? update(s) : s))})
}
return publishSettings({alerts: [...alerts, {url, notify, exceptions: []}]})
return publishSettings({alerts: [...alerts, update({url, notify: true, exceptions: []})]})
}
export const toggleRoomNotifications = (url: string, h: string) => {
const alerts = getSetting("alerts")
const existing = alerts.find(spec({url}))
export const setSpaceNotifications = (url: string, notify: boolean) =>
updateSpaceAlert(url, alert => ({...alert, notify, exceptions: []}))
if (existing) {
const exceptions = existing.exceptions.includes(h)
? remove(h, existing.exceptions)
: append(h, existing.exceptions)
export const toggleRoomNotifications = (url: string, h: string) =>
updateSpaceAlert(url, alert => ({...alert, exceptions: toggle(h, alert.exceptions)}))
return publishSettings({alerts: alerts.map(s => (s.url === url ? {...s, exceptions} : s))})
}
return publishSettings({alerts: [...alerts, {url, notify: true, exceptions: [h]}]})
}
export const toggleRoomMuted = (url: string, h: string) => {
const alerts = getSetting("alerts")
const existing = alerts.find(spec({url}))
if (existing) {
const current = existing.muted ?? []
const muted = current.includes(h) ? remove(h, current) : append(h, current)
return publishSettings({alerts: alerts.map(s => (s.url === url ? {...s, muted} : s))})
}
return publishSettings({alerts: [...alerts, {url, notify: true, exceptions: [], muted: [h]}]})
}
export const toggleRoomMuted = (url: string, h: string) =>
updateSpaceAlert(url, alert => ({...alert, muted: toggle(h, alert.muted ?? [])}))

View file

@ -1,8 +1,9 @@
import {derived, get, readable} from "svelte/store"
import {ago, avg, ms, now, spec} from "@welshman/lib"
import {makeHttpAuth, makeHttpAuthHeader} from "@welshman/util"
import {Logger} from "@welshman/app"
import type {LogMessage} from "@welshman/app"
import {fromApp} from "@app/core"
import {fromApp, user} from "@app/core"
import {navigate} from "@app/modal"
import {pushToast, toast} from "@app/toast"
@ -83,3 +84,33 @@ export const syncSignerAlerts = () =>
})
}
})
const httpAuthByUrl = new Map<
string,
{pubkey: string; expiresAt: number; header: Promise<string>}
>()
// A NIP-98 GET token for url, reused for maxAge seconds by the same pubkey where the server allows it.
export const getHttpAuthHeader = (url: string, maxAge = 0) => {
const {pubkey, signer} = user.get()
const cached = httpAuthByUrl.get(url)
if (cached?.pubkey === pubkey && cached.expiresAt > now()) {
return cached.header
}
const header = makeHttpAuth(url, "GET")
.then(template => signer.sign(template))
.then(makeHttpAuthHeader)
// A declined signature shouldn't be cached, or the user can't retry.
header.catch(() => {
if (httpAuthByUrl.get(url)?.header === header) {
httpAuthByUrl.delete(url)
}
})
httpAuthByUrl.set(url, {pubkey, expiresAt: now() + maxAge, header})
return header
}

View file

@ -1,12 +1,14 @@
import {get, writable} from "svelte/store"
import type {Maybe} from "@welshman/lib"
import type {TrustedEvent} from "@welshman/util"
import {toWav} from "@lib/audio"
import {errorMessage} from "@lib/util"
import Confirm from "@lib/components/Confirm.svelte"
import OpenRouterEnable from "@app/components/OpenRouterEnable.svelte"
import {endCall, isCallActive} from "@app/call"
import {profiles} from "@app/core"
import {pushModal} from "@app/modal"
import {postOpenRouter} from "@app/openrouter"
import {renderEventAsText} from "@app/render"
import {getSetting} from "@app/settings"
import {pushToast} from "@app/toast"
@ -21,10 +23,6 @@ const SPEECH_FORMAT = "mp3"
// Decoding resamples to the context's rate, so this is the rate the wav ends up at.
const SPEECH_RATE = 48000
const SPEECH_BIT_DEPTH = 16
const WAV_HEADER_LENGTH = 44
export type Speech = {
id: string
title: string
@ -37,70 +35,17 @@ export const speech = writable<Maybe<Speech>>(undefined)
const decode = (data: ArrayBuffer) =>
new OfflineAudioContext(1, 1, SPEECH_RATE).decodeAudioData(data)
const toWav = (audio: AudioBuffer) => {
const {numberOfChannels, sampleRate, length} = audio
const bytesPerFrame = (numberOfChannels * SPEECH_BIT_DEPTH) / 8
const dataLength = length * bytesPerFrame
const wav = new ArrayBuffer(WAV_HEADER_LENGTH + dataLength)
const view = new DataView(wav)
const ascii = (offset: number, value: string) => {
for (let i = 0; i < value.length; i++) {
view.setUint8(offset + i, value.charCodeAt(i))
}
}
ascii(0, "RIFF")
view.setUint32(4, 36 + dataLength, true)
ascii(8, "WAVEfmt ")
view.setUint32(16, 16, true)
view.setUint16(20, 1, true)
view.setUint16(22, numberOfChannels, true)
view.setUint32(24, sampleRate, true)
view.setUint32(28, sampleRate * bytesPerFrame, true)
view.setUint16(32, bytesPerFrame, true)
view.setUint16(34, SPEECH_BIT_DEPTH, true)
ascii(36, "data")
view.setUint32(40, dataLength, true)
const channels = Array.from({length: numberOfChannels}, (_, i) => audio.getChannelData(i))
let offset = WAV_HEADER_LENGTH
for (let frame = 0; frame < length; frame++) {
for (const samples of channels) {
// A decoded sample runs from -1 to 1, and the two ends of a signed 16 bit range differ in size.
const sample = Math.max(-1, Math.min(1, samples[frame]))
view.setInt16(offset, Math.round(sample * (sample < 0 ? 0x8000 : 0x7fff)), true)
offset += 2
}
}
return new Blob([wav], {type: "audio/wav"})
}
export const synthesize = async (text: string) => {
const response = await fetch("https://openrouter.ai/api/v1/audio/speech", {
method: "POST",
headers: {
Authorization: `Bearer ${getSetting("openrouter_key")}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
const response = await postOpenRouter(
"audio/speech",
JSON.stringify({
model: SPEECH_MODEL,
voice: SPEECH_VOICE,
input: text,
response_format: SPEECH_FORMAT,
}),
})
// A successful response is audio rather than json.
if (!response.ok) {
const {error} = await response.json().catch(() => ({error: undefined}))
throw new Error(error?.message || `OpenRouter returned a ${response.status}.`)
}
{"Content-Type": "application/json"},
)
return toWav(await decode(await response.arrayBuffer()))
}

View file

@ -48,7 +48,7 @@ import type {RepositoryUpdate, WrapItem} from "@welshman/net"
import {Relay, Zapper} from "@welshman/domain"
import type {RelayInfo, ZapperValues} from "@welshman/domain"
import {Handles, Plaintext, Relays, RelayStats, User, Zappers} from "@welshman/app"
import type {AppPolicy, IApp, RelayStatsItem} from "@welshman/app"
import type {AppPolicy, IApp, MapPlugin, RelayStatsItem} from "@welshman/app"
import {IDB} from "@lib/indexeddb"
import {appPolicies} from "@app/core"
import {DM_KINDS} from "@app/content"
@ -60,37 +60,51 @@ export const syncedWithGetter = <T>(config: SyncedConfig<T>) => {
return {...withGetter(store), ready: store.ready}
}
export const kv = call(() => {
type StringStorage = {
get: (key: string) => Promise<Maybe<string>>
set: (key: string, value: string) => Promise<void>
remove: (key: string) => Promise<void>
clear: () => Promise<void>
}
const makeJsonStore = (storage: StringStorage) => {
const enqueue = makeQueue()
const get = async <T>(key: string): Promise<T | undefined> => {
const result = await Preferences.get({key})
if (!result.value) {
return undefined
}
try {
return JSON.parse(result.value)
} catch (e) {
return undefined
const value = await storage.get(key)
if (value) {
try {
return JSON.parse(value)
} catch (e) {
return undefined
}
}
}
const set = async <T>(key: string, value: T): Promise<void> => {
await enqueue(() => Preferences.set({key, value: JSON.stringify(value)}))
await enqueue(() => storage.set(key, JSON.stringify(value)))
}
const remove = async (key: string) => {
await enqueue(() => Preferences.remove({key}))
await enqueue(() => storage.remove(key))
}
const clear = async () => {
await enqueue(() => Preferences.clear())
await enqueue(() => storage.clear())
}
return {get, set, remove, clear}
}
export const kv = makeJsonStore({
get: async key => (await Preferences.get({key})).value ?? undefined,
set: (key, value) => Preferences.set({key, value}),
remove: key => Preferences.remove({key}),
clear: () => Preferences.clear(),
})
const secretStorage =
const secretStorage: StringStorage =
Capacitor.getPlatform() === "electron"
? call(() => {
const plugin = registerPlugin<{
@ -101,14 +115,15 @@ const secretStorage =
}>("DesktopSecureStorage")
return {
get: async (key: string) => (await plugin.get({key})).value,
set: (key: string, value: string | undefined) =>
get: async key => (await plugin.get({key})).value,
set: (key, value) =>
value === undefined ? plugin.remove({key}) : plugin.set({key, value}),
remove: key => plugin.remove({key}),
clear: () => plugin.clear(),
}
})
: {
get: async (key: string) => {
get: async key => {
let value = await SecureStorage.getItem(key)
if (!value) {
@ -121,40 +136,16 @@ const secretStorage =
}
}
return value
return value ?? undefined
},
// Android's SecureStorage rejects undefined
set: (key: string, value: string | undefined) =>
set: (key, value) =>
value === undefined ? SecureStorage.removeItem(key) : SecureStorage.setItem(key, value),
remove: key => SecureStorage.removeItem(key),
clear: () => SecureStorage.clear(),
}
export const ss = call(() => {
const enqueue = makeQueue()
const get = async <T>(key: string): Promise<T | undefined> => {
const value = await secretStorage.get(key)
if (!value) {
return undefined
}
try {
return JSON.parse(value)
} catch (e) {
return undefined
}
}
const set = async <T>(key: string, value: T): Promise<void> => {
await enqueue(() => secretStorage.set(key, JSON.stringify(value)))
}
const clear = async () => {
await enqueue(() => secretStorage.clear())
}
return {get, set, clear}
})
export const ss = makeJsonStore(secretStorage)
const TABLES = [
{name: "events", keyPath: "id"},
@ -422,94 +413,72 @@ class Storage {
}
}
private initRelays = async () => {
const table = this.db.table<RelayInfo & {url: string}>("relays")
private persistMapPlugin = async <T, Row>(
name: string,
plugin: MapPlugin<T>,
fromRow: (row: Row) => Maybe<[string, T]>,
toRow: (key: string, item: T) => Row,
) => {
const table = this.db.table<Row>(name)
for (const row of await table.getAll()) {
this.app.use(Relays).set(row.url, new Relay(row.url, row))
const entry = fromRow(row)
if (entry) {
plugin.set(...entry)
}
}
const enqueue = batch(
FLUSH_INTERVAL,
idleWrite((relays: Relay[]) => table.bulkPut(relays.map(relay => ({...relay})))),
const enqueue = batch(FLUSH_INTERVAL, idleWrite(table.bulkPut))
return plugin.onItem((key, item) => {
if (item) {
enqueue(toRow(key, item))
}
})
}
private initRelays = () =>
this.persistMapPlugin(
"relays",
this.app.use(Relays),
(row: RelayInfo & {url: string}) => [row.url, new Relay(row.url, row)],
(_url, relay) => ({...relay}),
)
return this.app.use(Relays).onItem((_url, relay) => {
if (relay) {
enqueue(relay)
}
})
}
private initRelayStats = async () => {
const table = this.db.table<RelayStatsItem>("relayStats")
for (const row of await table.getAll()) {
this.app.use(RelayStats).set(row.url, row)
}
const enqueue = batch(FLUSH_INTERVAL, idleWrite(table.bulkPut))
return this.app.use(RelayStats).onItem((_url, stats) => {
if (stats) {
enqueue(stats)
}
})
}
private initHandles = async () => {
const table = this.db.table<Handle>("handles")
for (const row of await table.getAll()) {
this.app.use(Handles).set(row.nip05, row)
}
const enqueue = batch(FLUSH_INTERVAL, idleWrite(table.bulkPut))
return this.app.use(Handles).onItem((_nip05, handle) => {
if (handle) {
enqueue(handle)
}
})
}
private initZappers = async () => {
const table = this.db.table<ZapperValues>("zappers")
for (const row of await table.getAll()) {
// Validation is meaningless without these, and rows cached before they were required lack them.
if (row.pubkey && row.nostrPubkey) {
this.app.use(Zappers).set(row.lnurl, new Zapper(row))
}
}
const enqueue = batch(
FLUSH_INTERVAL,
idleWrite((zappers: Zapper[]) => table.bulkPut(zappers.map(zapper => ({...zapper})))),
private initRelayStats = () =>
this.persistMapPlugin(
"relayStats",
this.app.use(RelayStats),
(row: RelayStatsItem) => [row.url, row],
(_url, stats) => stats,
)
return this.app.use(Zappers).onItem((_lnurl, zapper) => {
if (zapper) {
enqueue(zapper)
}
})
}
private initHandles = () =>
this.persistMapPlugin(
"handles",
this.app.use(Handles),
(row: Handle) => [row.nip05, row],
(_nip05, handle) => handle,
)
private initPlaintext = async () => {
const table = this.db.table<PlaintextItem>("plaintext")
// Validation is meaningless without pubkey and nostrPubkey, and rows cached before they were required lack them.
private initZappers = () =>
this.persistMapPlugin(
"zappers",
this.app.use(Zappers),
(row: ZapperValues) =>
row.pubkey && row.nostrPubkey ? [row.lnurl, new Zapper(row)] : undefined,
(_lnurl, zapper) => ({...zapper}),
)
for (const {key, value} of await table.getAll()) {
this.app.use(Plaintext).set(key, value)
}
const enqueue = batch(FLUSH_INTERVAL, idleWrite(table.bulkPut))
return this.app.use(Plaintext).onItem((key, value) => {
if (value) {
enqueue({key, value})
}
})
}
private initPlaintext = () =>
this.persistMapPlugin(
"plaintext",
this.app.use(Plaintext),
(row: PlaintextItem) => [row.key, row.value],
(key, value) => ({key, value}),
)
private initWrapManager = async () => {
const table = this.db.table<WrapItem>("wrapManager")

View file

@ -91,6 +91,32 @@ const pullAndListen = ({url, signal, filters}: SyncOpts) => {
})
}
// Runs one sync per key, starting it when the key appears and stopping it when the key goes away.
const makeKeyedSyncs = (start: (key: string) => Unsubscriber) => {
const stopsByKey = new Map<string, Unsubscriber>()
const reconcile = (keys: Iterable<string>) => {
const keep = new Set(keys)
for (const [key, stop] of stopsByKey) {
if (!keep.has(key)) {
stopsByKey.delete(key)
stop()
}
}
for (const key of keep) {
if (!stopsByKey.has(key)) {
stopsByKey.set(key, start(key))
}
}
}
const stop = () => reconcile([])
return {reconcile, stop}
}
const userRelayList = deriveUserItem(RelayLists)
const userFollowList = deriveUserItem(FollowLists)
@ -157,25 +183,8 @@ const syncUserSpaceMembership = (url: string) => {
}
const syncUserData = () => {
const unsubscribersByUrl = new Map<string, Unsubscriber>()
const $pubkey = app.get().user?.pubkey
const syncSpaceMembership = ($activeSpaceUrls: string[]) => {
const urls = new Set($activeSpaceUrls)
for (const url of urls) {
if (!unsubscribersByUrl.has(url)) {
unsubscribersByUrl.set(url, syncUserSpaceMembership(url))
}
}
for (const [url, unsubscribe] of unsubscribersByUrl.entries()) {
if (!urls.has(url)) {
unsubscribersByUrl.delete(url)
unsubscribe()
}
}
}
const membershipSyncs = makeKeyedSyncs(syncUserSpaceMembership)
const syncUserLists = () => {
if ($pubkey) {
@ -207,12 +216,12 @@ const syncUserData = () => {
relayLists.get().load($pubkey)
}
const unsubscribeSpaceMembership = activeSpaceUrls.subscribe(syncSpaceMembership)
const unsubscribeSpaceMembership = activeSpaceUrls.subscribe(membershipSyncs.reconcile)
const unsubscribeRelayList = userRelayList.subscribe(syncUserLists)
const unsubscribeFollowList = userFollowList.subscribe(syncFollowNetwork)
return () => {
unsubscribersByUrl.forEach(call)
membershipSyncs.stop()
unsubscribeSpaceMembership()
unsubscribeRelayList()
unsubscribeFollowList()
@ -288,33 +297,15 @@ const syncSpaceDetail = (url: string) => {
}
const syncSpaces = () => {
const unsubscribersByUrl = new Map<string, Unsubscriber>()
const activitySyncs = makeKeyedSyncs(syncSpaceActivity)
let detailUrl: Maybe<string>
let unsubscribeDetail: Unsubscriber = noop
const unsubscribe = merged([activeSpaceUrls, page]).subscribe(([$activeSpaceUrls, $page]) => {
const urls = new Set($activeSpaceUrls)
const currentUrl = $page.params.relay ? decodeRelay($page.params.relay) : undefined
if (currentUrl) {
urls.add(currentUrl)
}
// Stop syncing removed spaces
for (const [url, unsubscribe] of unsubscribersByUrl.entries()) {
if (!urls.has(url)) {
unsubscribersByUrl.delete(url)
unsubscribe()
}
}
// Start syncing for new spaces
for (const url of urls) {
if (!unsubscribersByUrl.has(url)) {
unsubscribersByUrl.set(url, syncSpaceActivity(url))
}
}
activitySyncs.reconcile(currentUrl ? [...$activeSpaceUrls, currentUrl] : $activeSpaceUrls)
if (detailUrl !== currentUrl) {
detailUrl = currentUrl
@ -324,10 +315,7 @@ const syncSpaces = () => {
})
return () => {
for (const unsubscriber of unsubscribersByUrl.values()) {
unsubscriber()
}
activitySyncs.stop()
unsubscribeDetail()
unsubscribe()
}
@ -348,72 +336,37 @@ const syncDMRelay = (url: string, pubkey: string) => {
}
const syncDMs = () => {
const unsubscribersByUrl = new Map<string, Unsubscriber>()
const $pubkey = app.get().user?.pubkey
let currentPubkey: string | undefined
let currentShouldUnwrap = false
if ($pubkey) {
const relaySyncs = makeKeyedSyncs(url => syncDMRelay(url, $pubkey))
const unsubscribeAll = () => {
for (const [url, unsubscribe] of unsubscribersByUrl.entries()) {
unsubscribersByUrl.delete(url)
unsubscribe()
}
}
const subscribeAll = (pubkey: string, urls: string[]) => {
// Start syncing newly added relays
for (const url of urls) {
if (!unsubscribersByUrl.has(url)) {
unsubscribersByUrl.set(url, syncDMRelay(url, pubkey))
const syncList = () => {
if (shouldUnwrap.get()) {
relaySyncs.reconcile(messagingRelayLists.get().urls($pubkey).get())
}
}
// Stop syncing removed spaces
for (const [url, unsubscribe] of unsubscribersByUrl.entries()) {
if (!urls.includes(url)) {
unsubscribersByUrl.delete(url)
unsubscribe()
const loadList = async ($shouldUnwrap: boolean) => {
if ($shouldUnwrap) {
await relayLists.get().load($pubkey).catch(noop)
await messagingRelayLists.get().load($pubkey).catch(noop)
syncList()
}
}
}
const syncPubkey = async ($pubkey: Maybe<string>, $shouldUnwrap: boolean) => {
if ($pubkey !== currentPubkey) {
unsubscribeAll()
}
const unsubscribeUnwrap = shouldUnwrap.subscribe(loadList)
const unsubscribeList = userMessagingRelayList.subscribe(syncList)
currentPubkey = $pubkey
currentShouldUnwrap = $shouldUnwrap
if ($pubkey && $shouldUnwrap) {
await relayLists.get().load($pubkey).catch(noop)
await messagingRelayLists.get().load($pubkey).catch(noop)
if (currentPubkey === $pubkey && currentShouldUnwrap === $shouldUnwrap) {
subscribeAll($pubkey, messagingRelayLists.get().urls($pubkey).get())
}
return () => {
relaySyncs.stop()
unsubscribeUnwrap()
unsubscribeList()
}
}
const syncList = () => {
const $pubkey = app.get().user?.pubkey
if ($pubkey && shouldUnwrap.get()) {
subscribeAll($pubkey, messagingRelayLists.get().urls($pubkey).get())
}
}
const unsubscribeUser = merged([app, shouldUnwrap]).subscribe(([$app, $shouldUnwrap]) => {
syncPubkey($app.user?.pubkey, $shouldUnwrap)
})
const unsubscribeList = userMessagingRelayList.subscribe(syncList)
return () => {
unsubscribeAll()
unsubscribeUser()
unsubscribeList()
}
return noop
}
// Merge all synchronization functions

View file

@ -1,4 +1,6 @@
import {identity, uniq} from "@welshman/lib"
import {CLASSIFIED, EVENT_TIME, LONG_FORM, THREAD, ZAP_GOAL} from "@welshman/util"
import type {TrustedEvent} from "@welshman/util"
import {Article, Classified, Thread, ZapGoal, displayPubkey} from "@welshman/domain"
import {makeRoomKey} from "@welshman/app"
import {decodePubkey} from "@lib/util"
@ -33,13 +35,12 @@ const staticTitles = new Map<string, string>([
["/[bech32]", "Opening Link"],
])
const eventRoutes = new Set([
"/spaces/[relay]/threads/[id]",
"/spaces/[relay]/goals/[id]",
"/spaces/[relay]/calendar/[address]",
"/spaces/[relay]/classifieds/[address]",
"/spaces/[relay]/articles/[address]",
"/spaces/[relay]/polls/[id]",
const contentFallbackTitles = new Map<string, string>([
["/spaces/[relay]/threads/[id]", "Thread"],
["/spaces/[relay]/goals/[id]", "Goal"],
["/spaces/[relay]/calendar/[address]", "Event"],
["/spaces/[relay]/classifieds/[address]", "Listing"],
["/spaces/[relay]/articles/[address]", "Article"],
])
type RouteParams = Record<string, string | undefined>
@ -67,18 +68,19 @@ const getRoomTitle = (params: RouteParams) => {
return room?.meta?.name() || "Room"
}
const getEventForTitle = (routeId: string, params: RouteParams) => {
if (!eventRoutes.has(routeId)) {
return
const getContentTitle = (event: TrustedEvent) => {
switch (event.kind) {
case THREAD:
return reader(Thread)(event).title()
case EVENT_TIME:
return getEventTitle(event)
case CLASSIFIED:
return reader(Classified)(event).title()
case LONG_FORM:
return reader(Article)(event).title()
case ZAP_GOAL:
return event.content || reader(ZapGoal)(event).summary()
}
const eventId = params.id || params.address
if (!eventId) {
return
}
return app.get().repository.getEvent(eventId)
}
const getChatTitle = (chatId: string | undefined, pubkey: string | undefined) => {
@ -130,26 +132,12 @@ export const getPageTitle = ({page, pubkey}: PageTitleContext) => {
return makeTitle(getRoomTitle(page.params))
}
const event = getEventForTitle(routeId, page.params)
const contentFallbackTitle = contentFallbackTitles.get(routeId)
if (routeId === "/spaces/[relay]/threads/[id]") {
return makeTitle((event && reader(Thread)(event).title()) || "Thread")
}
if (contentFallbackTitle) {
const event = app.get().repository.getEvent(page.params.id || page.params.address || "")
if (routeId === "/spaces/[relay]/calendar/[address]") {
return makeTitle((event && getEventTitle(event)) || "Event")
}
if (routeId === "/spaces/[relay]/classifieds/[address]") {
return makeTitle((event && reader(Classified)(event).title()) || "Listing")
}
if (routeId === "/spaces/[relay]/articles/[address]") {
return makeTitle((event && reader(Article)(event).title()) || "Article")
}
if (routeId === "/spaces/[relay]/goals/[id]") {
return makeTitle(event?.content || (event && reader(ZapGoal)(event).summary()) || "Goal")
return makeTitle((event && getContentTitle(event)) || contentFallbackTitle)
}
return makeTitle()

89
src/lib/audio.ts Normal file
View file

@ -0,0 +1,89 @@
import {noop} from "@welshman/lib"
const WAV_BIT_DEPTH = 16
const WAV_HEADER_LENGTH = 44
export const toWav = (audio: AudioBuffer) => {
const {numberOfChannels, sampleRate, length} = audio
const bytesPerFrame = (numberOfChannels * WAV_BIT_DEPTH) / 8
const dataLength = length * bytesPerFrame
const wav = new ArrayBuffer(WAV_HEADER_LENGTH + dataLength)
const view = new DataView(wav)
const ascii = (offset: number, value: string) => {
for (let i = 0; i < value.length; i++) {
view.setUint8(offset + i, value.charCodeAt(i))
}
}
ascii(0, "RIFF")
view.setUint32(4, 36 + dataLength, true)
ascii(8, "WAVEfmt ")
view.setUint32(16, 16, true)
view.setUint16(20, 1, true)
view.setUint16(22, numberOfChannels, true)
view.setUint32(24, sampleRate, true)
view.setUint32(28, sampleRate * bytesPerFrame, true)
view.setUint16(32, bytesPerFrame, true)
view.setUint16(34, WAV_BIT_DEPTH, true)
ascii(36, "data")
view.setUint32(40, dataLength, true)
const channels = Array.from({length: numberOfChannels}, (_, i) => audio.getChannelData(i))
let offset = WAV_HEADER_LENGTH
for (let frame = 0; frame < length; frame++) {
for (const samples of channels) {
// A decoded sample runs from -1 to 1, and the two ends of a signed 16 bit range differ in size.
const sample = Math.max(-1, Math.min(1, samples[frame]))
view.setInt16(offset, Math.round(sample * (sample < 0 ? 0x8000 : 0x7fff)), true)
offset += 2
}
}
return new Blob([wav], {type: "audio/wav"})
}
export const listMediaDevices = async (): Promise<MediaDeviceInfo[]> => {
try {
return (await navigator.mediaDevices?.enumerateDevices?.()) ?? []
} catch {
return []
}
}
// Calls onFrame with the root mean square of the stream's waveform, from 0 to 1, on every animation frame.
export const watchAudioLevel = (stream: MediaStream, onFrame: (rms: number) => void) => {
const context = new AudioContext()
const analyser = context.createAnalyser()
analyser.fftSize = 512
context.createMediaStreamSource(stream).connect(analyser)
const samples = new Uint8Array(analyser.frequencyBinCount)
const measure = () => {
analyser.getByteTimeDomainData(samples)
let sumSquares = 0
for (const sample of samples) {
sumSquares += ((sample - 128) / 128) ** 2
}
onFrame(Math.sqrt(sumSquares / samples.length))
frame = requestAnimationFrame(measure)
}
let frame = requestAnimationFrame(measure)
return () => {
cancelAnimationFrame(frame)
context.close().catch(noop)
}
}

174
src/lib/tileGrid.ts Normal file
View file

@ -0,0 +1,174 @@
/** The lower aspect bound is 1:1 on landscape and 3:4 on portrait, and the upper is 16:9. */
const TILE_ASPECT_PORTRAIT = 3 / 4
const TILE_ASPECT_LANDSCAPE = 16 / 9
const TILE_GAP = 8
/** Below this the grid scrolls rather than forcing portrait tiles, which is ~6-8 tiles on a portrait phone. */
const MIN_TILE_HEIGHT = 120
export type TileRow = {
columnCount: number
tileWidth: number
tileHeight: number
/** Total width of this row including gaps — for the template's max-width */
rowWidth: number
aspectRatio: number
}
/** All tiles share one size. Full rows fill the container width and a partial row is centered. */
export type AdaptiveTileGrid = {
rows: TileRow[]
totalWidth: number
totalHeight: number
}
/** Score for comparing candidate layouts. Lower is better. */
type LayoutScore = {
/** Penalty for vertical overflow: 0 if none, else huge */
verticalOverflowPenalty: number
/** Penalty for horizontal overflow: 0 if none, else huge */
horizontalOverflowPenalty: number
/** Penalty when tiles fall below MIN_TILE_HEIGHT even while fitting */
minTileHeightPenalty: number
/** Negative tile area (we want to maximize area, so negate) */
negativeArea: number
/** Leftover whitespace (underflow) in px */
whitespace: number
/** Average absolute deviation from 16:9 across all tiles */
aspectDeviation: number
}
const compareScores = (a: LayoutScore, b: LayoutScore): number => {
const fields: (keyof LayoutScore)[] = [
"verticalOverflowPenalty",
"horizontalOverflowPenalty",
"minTileHeightPenalty",
"negativeArea",
"whitespace",
"aspectDeviation",
]
for (const field of fields) {
if (a[field] !== b[field]) {
return a[field] - b[field]
}
}
return 0
}
/** The largest tile that fits, shrunk to whichever dimension is more constraining. */
const fitTile = (availWidth: number, availHeight: number, minAspect: number) => {
const fillAspect = availWidth / availHeight
const aspectRatio = Math.max(minAspect, Math.min(TILE_ASPECT_LANDSCAPE, fillAspect))
if (fillAspect >= aspectRatio) {
const tileWidth = availHeight * aspectRatio
return {tileWidth, tileHeight: availHeight, aspectRatio}
}
const tileHeight = availWidth / aspectRatio
return {tileWidth: availWidth, tileHeight, aspectRatio}
}
/** A candidate grid for a given column count, with the partial last row centered. */
const buildCandidate = (
tileCount: number,
columnCount: number,
containerWidth: number,
containerHeight: number,
minAspect: number,
): AdaptiveTileGrid | undefined => {
const gap = TILE_GAP
const fullRowCount = Math.floor(tileCount / columnCount)
const remainder = tileCount % columnCount
const totalRowCount = fullRowCount + (remainder > 0 ? 1 : 0)
const availHeight = (containerHeight - (totalRowCount - 1) * gap) / totalRowCount
const availWidth = (containerWidth - (columnCount - 1) * gap) / columnCount
if (availWidth <= 0 || availHeight <= 0) {
return undefined
}
const {tileWidth, tileHeight, aspectRatio} = fitTile(availWidth, availHeight, minAspect)
const rows: TileRow[] = []
for (let r = 0; r < fullRowCount; r++) {
rows.push({
columnCount,
tileWidth,
tileHeight,
rowWidth: columnCount * tileWidth + (columnCount - 1) * gap,
aspectRatio,
})
}
if (remainder > 0) {
rows.push({
columnCount: remainder,
tileWidth,
tileHeight,
rowWidth: remainder * tileWidth + (remainder - 1) * gap,
aspectRatio,
})
}
const totalHeight = totalRowCount * tileHeight + (totalRowCount - 1) * gap
const totalWidth = columnCount * tileWidth + (columnCount - 1) * gap
return {rows, totalWidth, totalHeight}
}
/** Scored on overflow first, then whitespace, then distance from 16:9, then tile size. */
export const computeAdaptiveGrid = (
tileCount: number,
containerWidth: number,
containerHeight: number,
): AdaptiveTileGrid | undefined => {
if (tileCount <= 0 || containerWidth <= 0 || containerHeight <= 0) {
return undefined
}
const minAspect = containerWidth / containerHeight >= 4 / 3 ? 1 : TILE_ASPECT_PORTRAIT
let best: AdaptiveTileGrid | undefined
let bestScore: LayoutScore | undefined
for (let columnCount = 1; columnCount <= tileCount; columnCount++) {
const candidate = buildCandidate(
tileCount,
columnCount,
containerWidth,
containerHeight,
minAspect,
)
if (!candidate) {
continue
}
const {tileWidth, tileHeight, aspectRatio} = candidate.rows[0]
const verticalOverflow = Math.max(0, candidate.totalHeight - containerHeight)
const horizontalOverflow = Math.max(0, candidate.totalWidth - containerWidth)
const whitespace =
Math.max(0, containerWidth - candidate.totalWidth) +
Math.max(0, containerHeight - candidate.totalHeight)
const totalArea = tileWidth * tileHeight * tileCount
const overflowPenaltyMultiplier = tileHeight >= MIN_TILE_HEIGHT ? 1_000_000 : 1
const verticalOverflowPenalty = verticalOverflow * overflowPenaltyMultiplier
const horizontalOverflowPenalty = horizontalOverflow * overflowPenaltyMultiplier
const minTileHeightPenalty =
tileHeight < MIN_TILE_HEIGHT ? (MIN_TILE_HEIGHT - tileHeight) * 1000 : 0
const score: LayoutScore = {
verticalOverflowPenalty,
horizontalOverflowPenalty,
minTileHeightPenalty,
negativeArea: -totalArea,
whitespace,
aspectDeviation: Math.abs(aspectRatio - TILE_ASPECT_LANDSCAPE),
}
if (!bestScore || compareScores(score, bestScore) < 0) {
bestScore = score
best = candidate
}
}
return best
}