feat: добавить базовые сервисы image-platform
- добавлены backend, admin, gateway и worker skeleton - добавлены Drizzle schema, database package и initial migration - добавлены shared packages для RabbitMQ topology и S3 helpers - обновлены dev-инфраструктура, env example, scripts и dependencies - обновлена документация под versioned image URLs и read-through flow
This commit is contained in:
33
.env.example
33
.env.example
@@ -19,12 +19,39 @@ S3_SECRET_ACCESS_KEY=image-password
|
||||
S3_FORCE_PATH_STYLE=true
|
||||
|
||||
# Future local services
|
||||
PUBLIC_API_BASE_URL=http://localhost:3001
|
||||
BACKEND_PORT=3001
|
||||
ADMIN_PORT=5173
|
||||
GATEWAY_HOST=0.0.0.0
|
||||
GATEWAY_PORT=8888
|
||||
PUBLIC_BACKEND_BASE_URL=http://localhost:3001
|
||||
PUBLIC_ADMIN_BASE_URL=http://localhost:5173
|
||||
PUBLIC_IMAGE_BASE_URL=http://localhost:8888
|
||||
|
||||
# imgproxy is always external for image-platform.
|
||||
# Local example: run imgproxy separately on localhost:18080.
|
||||
# Gateway proxies /api and Swagger routes to this upstream.
|
||||
GATEWAY_BACKEND_UPSTREAM=http://localhost:3001
|
||||
|
||||
# Dev imgproxy is exposed only on localhost.
|
||||
IMGPROXY_PORT=18080
|
||||
IMGPROXY_UPSTREAM=http://localhost:18080
|
||||
IMGPROXY_SIGNING_ENABLED=false
|
||||
IMGPROXY_KEY=
|
||||
IMGPROXY_SALT=
|
||||
IMGPROXY_WORKERS=2
|
||||
IMGPROXY_MAX_SRC_RESOLUTION=20
|
||||
IMGPROXY_DOWNLOAD_TIMEOUT=30
|
||||
IMGPROXY_ALLOWED_SOURCES=
|
||||
|
||||
# RabbitMQ dev broker is exposed only on localhost.
|
||||
RABBITMQ_DEFAULT_USER=image
|
||||
RABBITMQ_DEFAULT_PASS=image-password
|
||||
RABBITMQ_DEFAULT_VHOST=image_platform
|
||||
RABBITMQ_PORT=5672
|
||||
RABBITMQ_MANAGEMENT_PORT=15672
|
||||
RABBITMQ_URL=amqp://image:image-password@localhost:5672/image_platform
|
||||
WORKER_PREFETCH=2
|
||||
|
||||
# Queue topology
|
||||
RABBITMQ_JOBS_EXCHANGE=image-platform.jobs
|
||||
RABBITMQ_GENERATE_VARIANT_QUEUE=image.generate-variant
|
||||
RABBITMQ_GENERATE_VARIANT_DLX=image-platform.jobs.dlx
|
||||
RABBITMQ_GENERATE_VARIANT_DLQ=image.generate-variant.dlq
|
||||
|
||||
38
README.md
38
README.md
@@ -4,16 +4,18 @@ Image Platform - отдельная площадка для управления
|
||||
|
||||
## Статус
|
||||
|
||||
Сейчас создан только базовый monorepo и dev-инфраструктура. Приложения `api`, `admin` и `gateway` пока намеренно не созданы.
|
||||
Сейчас создан базовый monorepo, dev-инфраструктура, NestJS backend с Swagger, чистый Vite React TS admin, Fastify gateway skeleton, Drizzle database package, shared queue/storage packages и worker skeleton.
|
||||
|
||||
## Целевая схема
|
||||
|
||||
```text
|
||||
client
|
||||
-> CDN optional
|
||||
-> gateway Caddy/Souin hot cache
|
||||
-> Fastify gateway + L1 memory cache
|
||||
-> NestJS backend
|
||||
-> S3/Object Storage persistent variants
|
||||
-> generator/worker
|
||||
-> generator/worker on miss
|
||||
-> RabbitMQ
|
||||
-> external imgproxy
|
||||
-> source/original image
|
||||
```
|
||||
@@ -27,20 +29,28 @@ client
|
||||
- PostgreSQL
|
||||
- MinIO
|
||||
- MinIO bucket init
|
||||
- imgproxy dev instance
|
||||
- RabbitMQ
|
||||
|
||||
Позже нодой будут запускаться:
|
||||
Нодой запускается:
|
||||
|
||||
- NestJS API
|
||||
- worker
|
||||
- NestJS backend
|
||||
- React/Vite admin
|
||||
- Fastify gateway
|
||||
- worker
|
||||
|
||||
Gateway будет добавлен отдельно позже.
|
||||
Gateway уже добавлен как JS/Fastify skeleton. Сейчас `/images/*` возвращает `501`, пока не подключены DB/S3/imgproxy.
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
pnpm install
|
||||
pnpm infra:up
|
||||
pnpm db:migrate
|
||||
pnpm infra:config
|
||||
pnpm backend:dev
|
||||
pnpm admin:dev
|
||||
pnpm gateway:dev
|
||||
pnpm worker:dev
|
||||
```
|
||||
|
||||
Порты по умолчанию:
|
||||
@@ -50,11 +60,21 @@ pnpm infra:config
|
||||
| PostgreSQL | `localhost:5433` |
|
||||
| MinIO API | `http://localhost:9000` |
|
||||
| MinIO Console | `http://localhost:9001` |
|
||||
| imgproxy | `http://localhost:18080` |
|
||||
| RabbitMQ | `amqp://localhost:5672` |
|
||||
| RabbitMQ Management | `http://localhost:15672` |
|
||||
| Backend API | `http://localhost:3001/api` |
|
||||
| Swagger | `http://localhost:3001/docs` |
|
||||
| Admin | `http://localhost:5173` |
|
||||
| Gateway | `http://localhost:8888` |
|
||||
|
||||
Если старый локальный `image-gateway` уже занимает `8888`, остановите его или задайте другой `GATEWAY_PORT` в `.env`.
|
||||
|
||||
## Документация
|
||||
|
||||
- `docs/architecture.md` - целевая архитектура и ответственность компонентов.
|
||||
- `docs/development.md` - локальный dev flow.
|
||||
- `docs/data-model.md` - черновик PostgreSQL модели.
|
||||
- `docs/api-contract-draft.md` - черновик будущего JSON API.
|
||||
- `docs/data-model.md` - текущая Drizzle/PostgreSQL модель.
|
||||
- `docs/backend-contract-draft.md` - черновик будущего backend-контракта.
|
||||
- `docs/imgproxy-contract.md` - контракт с external imgproxy.
|
||||
- `docs/next-image-provider.md` - контракт custom provider для `next/image`.
|
||||
|
||||
12
apps/admin/index.html
Normal file
12
apps/admin/index.html
Normal file
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="ru">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Image Platform Admin</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
24
apps/admin/package.json
Normal file
24
apps/admin/package.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "@image-platform/admin",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"build": "tsc -b && vite build",
|
||||
"dev": "vite --host 0.0.0.0 --port 5173",
|
||||
"preview": "vite preview --host 0.0.0.0 --port 5173",
|
||||
"typecheck": "tsc -b"
|
||||
},
|
||||
"dependencies": {
|
||||
"react": "^19.2.5",
|
||||
"react-dom": "^19.2.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.12.2",
|
||||
"@types/react": "^19.2.14",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"@vitejs/plugin-react": "^6.0.1",
|
||||
"typescript": "^5.9.3",
|
||||
"vite": "^8.0.10"
|
||||
}
|
||||
}
|
||||
111
apps/admin/src/App.css
Normal file
111
apps/admin/src/App.css
Normal file
@@ -0,0 +1,111 @@
|
||||
:root {
|
||||
color: #171411;
|
||||
background: #f7f4ee;
|
||||
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
font-synthesis: none;
|
||||
text-rendering: optimizeLegibility;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
min-width: 320px;
|
||||
min-height: 100vh;
|
||||
margin: 0;
|
||||
background:
|
||||
radial-gradient(circle at 18% 18%, rgba(123, 76, 255, 0.18), transparent 30rem),
|
||||
#f7f4ee;
|
||||
}
|
||||
|
||||
button,
|
||||
input,
|
||||
textarea,
|
||||
select {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
.app-shell {
|
||||
min-height: 100vh;
|
||||
padding: 48px 24px;
|
||||
}
|
||||
|
||||
.hero {
|
||||
max-width: 960px;
|
||||
margin: 0 auto;
|
||||
padding: 40px;
|
||||
border: 1px solid #e4ded4;
|
||||
border-radius: 32px;
|
||||
background: rgba(255, 255, 255, 0.82);
|
||||
box-shadow: 0 22px 80px rgba(40, 32, 21, 0.08);
|
||||
}
|
||||
|
||||
.eyebrow {
|
||||
margin: 0 0 16px;
|
||||
color: #7b4cff;
|
||||
font-size: 13px;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.22em;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
h1 {
|
||||
max-width: 760px;
|
||||
margin: 0;
|
||||
font-size: clamp(40px, 7vw, 76px);
|
||||
line-height: 0.92;
|
||||
letter-spacing: -0.06em;
|
||||
}
|
||||
|
||||
.lead {
|
||||
max-width: 680px;
|
||||
margin: 24px 0 0;
|
||||
color: #73695d;
|
||||
font-size: 18px;
|
||||
line-height: 1.7;
|
||||
}
|
||||
|
||||
.cards {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||
gap: 16px;
|
||||
max-width: 960px;
|
||||
margin: 24px auto 0;
|
||||
}
|
||||
|
||||
.card {
|
||||
min-height: 150px;
|
||||
padding: 24px;
|
||||
border: 1px solid #e4ded4;
|
||||
border-radius: 24px;
|
||||
background: rgba(255, 255, 255, 0.76);
|
||||
}
|
||||
|
||||
.card h2 {
|
||||
margin: 0;
|
||||
font-size: 18px;
|
||||
}
|
||||
|
||||
.card p {
|
||||
margin: 12px 0 0;
|
||||
color: #73695d;
|
||||
line-height: 1.55;
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
.app-shell {
|
||||
padding: 24px 16px;
|
||||
}
|
||||
|
||||
.hero {
|
||||
padding: 28px;
|
||||
border-radius: 24px;
|
||||
}
|
||||
|
||||
.cards {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
40
apps/admin/src/App.tsx
Normal file
40
apps/admin/src/App.tsx
Normal file
@@ -0,0 +1,40 @@
|
||||
import "./App.css"
|
||||
|
||||
const cards = [
|
||||
{
|
||||
title: "Assets",
|
||||
description: "Каталог исходных изображений и связанной metadata.",
|
||||
},
|
||||
{
|
||||
title: "Variants",
|
||||
description: "Будущие AVIF/WebP/JPEG variants, presets и статусы генерации.",
|
||||
},
|
||||
{
|
||||
title: "Storage",
|
||||
description: "PostgreSQL как source of truth, S3/MinIO как хранилище bytes.",
|
||||
},
|
||||
]
|
||||
|
||||
export function App() {
|
||||
return (
|
||||
<main className="app-shell">
|
||||
<section className="hero">
|
||||
<p className="eyebrow">Image Platform Admin</p>
|
||||
<h1>чистый Vite React TS app</h1>
|
||||
<p className="lead">
|
||||
Это стартовая админка без UI-фреймворков. Дальше сюда добавим управление allowed hosts,
|
||||
assets, variants и presets.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section className="cards" aria-label="Будущие разделы">
|
||||
{cards.map((card) => (
|
||||
<article className="card" key={card.title}>
|
||||
<h2>{card.title}</h2>
|
||||
<p>{card.description}</p>
|
||||
</article>
|
||||
))}
|
||||
</section>
|
||||
</main>
|
||||
)
|
||||
}
|
||||
10
apps/admin/src/main.tsx
Normal file
10
apps/admin/src/main.tsx
Normal file
@@ -0,0 +1,10 @@
|
||||
import { StrictMode } from "react"
|
||||
import { createRoot } from "react-dom/client"
|
||||
|
||||
import { App } from "./App"
|
||||
|
||||
createRoot(document.getElementById("root")!).render(
|
||||
<StrictMode>
|
||||
<App />
|
||||
</StrictMode>,
|
||||
)
|
||||
22
apps/admin/tsconfig.app.json
Normal file
22
apps/admin/tsconfig.app.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"composite": true,
|
||||
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
|
||||
"target": "ES2022",
|
||||
"useDefineForClassFields": true,
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||
"allowJs": false,
|
||||
"skipLibCheck": true,
|
||||
"esModuleInterop": true,
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"strict": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"noEmit": true,
|
||||
"jsx": "react-jsx"
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
11
apps/admin/tsconfig.json
Normal file
11
apps/admin/tsconfig.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"files": [],
|
||||
"references": [
|
||||
{
|
||||
"path": "./tsconfig.app.json"
|
||||
},
|
||||
{
|
||||
"path": "./tsconfig.node.json"
|
||||
}
|
||||
]
|
||||
}
|
||||
18
apps/admin/tsconfig.node.json
Normal file
18
apps/admin/tsconfig.node.json
Normal file
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"composite": true,
|
||||
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
|
||||
"target": "ES2023",
|
||||
"lib": ["ES2023"],
|
||||
"allowJs": false,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"isolatedModules": true,
|
||||
"noEmit": true,
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["vite.config.ts"]
|
||||
}
|
||||
6
apps/admin/vite.config.ts
Normal file
6
apps/admin/vite.config.ts
Normal file
@@ -0,0 +1,6 @@
|
||||
import react from "@vitejs/plugin-react"
|
||||
import { defineConfig } from "vite"
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [react()],
|
||||
})
|
||||
8
apps/backend/nest-cli.json
Normal file
8
apps/backend/nest-cli.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/nest-cli",
|
||||
"collection": "@nestjs/schematics",
|
||||
"sourceRoot": "src",
|
||||
"compilerOptions": {
|
||||
"deleteOutDir": true
|
||||
}
|
||||
}
|
||||
27
apps/backend/package.json
Normal file
27
apps/backend/package.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "@image-platform/backend",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"build": "nest build",
|
||||
"dev": "nest start --watch",
|
||||
"start": "node dist/main.js",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"@nestjs/common": "^11.0.0",
|
||||
"@nestjs/core": "^11.0.0",
|
||||
"@nestjs/platform-express": "^11.0.0",
|
||||
"@nestjs/swagger": "^11.0.0",
|
||||
"reflect-metadata": "^0.2.2",
|
||||
"rxjs": "^7.8.1",
|
||||
"swagger-ui-express": "^5.0.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@nestjs/cli": "^11.0.0",
|
||||
"@nestjs/schematics": "^11.0.0",
|
||||
"@types/node": "^24.0.0",
|
||||
"@types/swagger-ui-express": "^4.1.8",
|
||||
"typescript": "^5.9.0"
|
||||
}
|
||||
}
|
||||
9
apps/backend/src/app.module.ts
Normal file
9
apps/backend/src/app.module.ts
Normal file
@@ -0,0 +1,9 @@
|
||||
import { Module } from "@nestjs/common"
|
||||
|
||||
import { HealthController } from "./health/health.controller"
|
||||
import { InternalImagesController } from "./internal-images/internal-images.controller"
|
||||
|
||||
@Module({
|
||||
controllers: [HealthController, InternalImagesController],
|
||||
})
|
||||
export class AppModule {}
|
||||
9
apps/backend/src/health/health-response.dto.ts
Normal file
9
apps/backend/src/health/health-response.dto.ts
Normal file
@@ -0,0 +1,9 @@
|
||||
import { ApiProperty } from "@nestjs/swagger"
|
||||
|
||||
export class HealthResponseDto {
|
||||
@ApiProperty({ example: "image-platform-api" })
|
||||
service!: string
|
||||
|
||||
@ApiProperty({ example: "ok" })
|
||||
status!: string
|
||||
}
|
||||
18
apps/backend/src/health/health.controller.ts
Normal file
18
apps/backend/src/health/health.controller.ts
Normal file
@@ -0,0 +1,18 @@
|
||||
import { Controller, Get } from "@nestjs/common"
|
||||
import { ApiOkResponse, ApiOperation, ApiTags } from "@nestjs/swagger"
|
||||
|
||||
import { HealthResponseDto } from "./health-response.dto"
|
||||
|
||||
@ApiTags("system")
|
||||
@Controller("health")
|
||||
export class HealthController {
|
||||
@Get()
|
||||
@ApiOperation({ summary: "Проверить состояние API" })
|
||||
@ApiOkResponse({ type: HealthResponseDto })
|
||||
getHealth(): HealthResponseDto {
|
||||
return {
|
||||
service: "image-platform-api",
|
||||
status: "ok",
|
||||
}
|
||||
}
|
||||
}
|
||||
21
apps/backend/src/internal-images/ensure-image-variant.dto.ts
Normal file
21
apps/backend/src/internal-images/ensure-image-variant.dto.ts
Normal file
@@ -0,0 +1,21 @@
|
||||
import { ApiProperty } from "@nestjs/swagger"
|
||||
|
||||
export class EnsureImageVariantRequestDto {
|
||||
@ApiProperty({ example: "asset_123" })
|
||||
assetId!: string
|
||||
|
||||
@ApiProperty({ example: 4, minimum: 1 })
|
||||
version!: number
|
||||
|
||||
@ApiProperty({ example: "card" })
|
||||
preset!: string
|
||||
|
||||
@ApiProperty({ example: 640, minimum: 1 })
|
||||
width!: number
|
||||
|
||||
@ApiProperty({ example: 80, minimum: 1 })
|
||||
quality!: number
|
||||
|
||||
@ApiProperty({ enum: ["auto", "avif", "webp", "jpg", "png"], example: "auto" })
|
||||
format!: "auto" | "avif" | "jpg" | "png" | "webp"
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
import { Body, Controller, NotImplementedException, Post } from "@nestjs/common"
|
||||
import { ApiOperation, ApiResponse, ApiTags } from "@nestjs/swagger"
|
||||
|
||||
import { EnsureImageVariantRequestDto } from "./ensure-image-variant.dto"
|
||||
|
||||
@ApiTags("internal-images")
|
||||
@Controller("internal/images")
|
||||
export class InternalImagesController {
|
||||
@Post("ensure")
|
||||
@ApiOperation({ summary: "Ensure image variant for Gateway L1 miss" })
|
||||
@ApiResponse({ status: 501, description: "Read-through image pipeline is not implemented yet" })
|
||||
ensureImageVariant(@Body() request: EnsureImageVariantRequestDto): never {
|
||||
throw new NotImplementedException({
|
||||
message: "image read-through pipeline is not implemented yet",
|
||||
request,
|
||||
status: "not_implemented",
|
||||
})
|
||||
}
|
||||
}
|
||||
37
apps/backend/src/main.ts
Normal file
37
apps/backend/src/main.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
import { NestFactory } from "@nestjs/core"
|
||||
import { DocumentBuilder, SwaggerModule } from "@nestjs/swagger"
|
||||
|
||||
import { AppModule } from "./app.module"
|
||||
|
||||
async function bootstrap() {
|
||||
const app = await NestFactory.create(AppModule)
|
||||
|
||||
app.setGlobalPrefix("api")
|
||||
app.enableShutdownHooks()
|
||||
|
||||
const openApiConfig = new DocumentBuilder()
|
||||
.setTitle("Image Platform API")
|
||||
.setDescription("Control plane for image assets, variants, S3 storage and external imgproxy.")
|
||||
.setVersion("0.1.0")
|
||||
.addTag("system")
|
||||
.addTag("assets")
|
||||
.addTag("variants")
|
||||
.addTag("allowed-hosts")
|
||||
.addTag("internal-images")
|
||||
.build()
|
||||
|
||||
const openApiDocument = SwaggerModule.createDocument(app, openApiConfig)
|
||||
|
||||
SwaggerModule.setup("docs", app, openApiDocument, {
|
||||
jsonDocumentUrl: "docs-json",
|
||||
swaggerOptions: {
|
||||
persistAuthorization: true,
|
||||
},
|
||||
})
|
||||
|
||||
const port = Number.parseInt(process.env.API_PORT ?? "3001", 10)
|
||||
|
||||
await app.listen(port)
|
||||
}
|
||||
|
||||
void bootstrap()
|
||||
8
apps/backend/tsconfig.build.json
Normal file
8
apps/backend/tsconfig.build.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"declaration": true,
|
||||
"noEmit": false
|
||||
},
|
||||
"exclude": ["node_modules", "dist", "test", "**/*.spec.ts"]
|
||||
}
|
||||
23
apps/backend/tsconfig.json
Normal file
23
apps/backend/tsconfig.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"baseUrl": "./",
|
||||
"declaration": true,
|
||||
"emitDecoratorMetadata": true,
|
||||
"esModuleInterop": true,
|
||||
"experimentalDecorators": true,
|
||||
"incremental": true,
|
||||
"module": "commonjs",
|
||||
"moduleResolution": "node",
|
||||
"outDir": "./dist",
|
||||
"removeComments": true,
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"strictPropertyInitialization": false,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
20
apps/gateway/package.json
Normal file
20
apps/gateway/package.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"name": "@image-platform/gateway",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"dev": "tsx watch src/main.ts",
|
||||
"start": "node dist/main.js",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"fastify": "^5.8.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.6.0",
|
||||
"tsx": "^4.21.0",
|
||||
"typescript": "^6.0.3"
|
||||
}
|
||||
}
|
||||
27
apps/gateway/src/config.ts
Normal file
27
apps/gateway/src/config.ts
Normal file
@@ -0,0 +1,27 @@
|
||||
export type GatewayConfig = {
|
||||
backendUpstream: URL
|
||||
host: string
|
||||
port: number
|
||||
}
|
||||
|
||||
export function loadGatewayConfig(): GatewayConfig {
|
||||
return {
|
||||
backendUpstream: new URL(process.env.GATEWAY_BACKEND_UPSTREAM ?? "http://localhost:3001"),
|
||||
host: process.env.GATEWAY_HOST ?? "0.0.0.0",
|
||||
port: parsePort(process.env.GATEWAY_PORT, 8888),
|
||||
}
|
||||
}
|
||||
|
||||
function parsePort(value: string | undefined, fallback: number) {
|
||||
if (!value) {
|
||||
return fallback
|
||||
}
|
||||
|
||||
const parsed = Number.parseInt(value, 10)
|
||||
|
||||
if (!Number.isFinite(parsed) || parsed < 1 || parsed > 65535) {
|
||||
throw new Error(`Invalid port: ${value}`)
|
||||
}
|
||||
|
||||
return parsed
|
||||
}
|
||||
12
apps/gateway/src/main.ts
Normal file
12
apps/gateway/src/main.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
import { loadGatewayConfig } from "./config.js"
|
||||
import { createGatewayServer } from "./server.js"
|
||||
|
||||
const config = loadGatewayConfig()
|
||||
const app = createGatewayServer(config)
|
||||
|
||||
try {
|
||||
await app.listen({ host: config.host, port: config.port })
|
||||
} catch (error) {
|
||||
app.log.error(error)
|
||||
process.exit(1)
|
||||
}
|
||||
77
apps/gateway/src/proxy.ts
Normal file
77
apps/gateway/src/proxy.ts
Normal file
@@ -0,0 +1,77 @@
|
||||
import { Readable } from "node:stream"
|
||||
|
||||
import type { FastifyReply, FastifyRequest } from "fastify"
|
||||
|
||||
const hopByHopHeaders = new Set([
|
||||
"connection",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"te",
|
||||
"trailer",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
])
|
||||
|
||||
export async function proxyToUpstream(request: FastifyRequest, reply: FastifyReply, upstreamBaseUrl: URL) {
|
||||
const upstreamUrl = new URL(request.url, upstreamBaseUrl)
|
||||
const headers = buildProxyHeaders(request)
|
||||
const init: RequestInit & { duplex?: "half" } = {
|
||||
headers,
|
||||
method: request.method,
|
||||
redirect: "manual",
|
||||
}
|
||||
|
||||
if (request.method !== "GET" && request.method !== "HEAD") {
|
||||
init.body = request.raw as RequestInit["body"]
|
||||
init.duplex = "half"
|
||||
}
|
||||
|
||||
const response = await fetch(upstreamUrl, init)
|
||||
|
||||
reply.code(response.status)
|
||||
|
||||
response.headers.forEach((value, key) => {
|
||||
if (!hopByHopHeaders.has(key.toLowerCase())) {
|
||||
reply.header(key, value)
|
||||
}
|
||||
})
|
||||
|
||||
if (!response.body) {
|
||||
return reply.send()
|
||||
}
|
||||
|
||||
return reply.send(Readable.fromWeb(response.body))
|
||||
}
|
||||
|
||||
function buildProxyHeaders(request: FastifyRequest) {
|
||||
const headers = new Headers()
|
||||
|
||||
for (const [key, rawValue] of Object.entries(request.headers)) {
|
||||
const lowerKey = key.toLowerCase()
|
||||
|
||||
if (lowerKey === "host" || hopByHopHeaders.has(lowerKey)) {
|
||||
continue
|
||||
}
|
||||
|
||||
if (Array.isArray(rawValue)) {
|
||||
for (const value of rawValue) {
|
||||
headers.append(key, value)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (typeof rawValue === "string") {
|
||||
headers.set(key, rawValue)
|
||||
}
|
||||
}
|
||||
|
||||
headers.set("x-forwarded-host", request.headers.host ?? "")
|
||||
headers.set("x-forwarded-proto", "http")
|
||||
|
||||
if (request.ip) {
|
||||
headers.set("x-forwarded-for", request.ip)
|
||||
}
|
||||
|
||||
return headers
|
||||
}
|
||||
90
apps/gateway/src/server.ts
Normal file
90
apps/gateway/src/server.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
import Fastify from "fastify"
|
||||
|
||||
import type { GatewayConfig } from "./config.js"
|
||||
import { proxyToUpstream } from "./proxy.js"
|
||||
|
||||
export function createGatewayServer(config: GatewayConfig) {
|
||||
const app = Fastify({
|
||||
logger: true,
|
||||
})
|
||||
|
||||
app.get("/health", async () => ({
|
||||
service: "image-platform-gateway",
|
||||
status: "ok",
|
||||
}))
|
||||
|
||||
app.get<{ Params: ImageParams; Querystring: ImageQuery }>(
|
||||
"/images/:assetId/:version/:preset",
|
||||
async (request, reply) => {
|
||||
const version = parseVersionParam(request.params.version)
|
||||
|
||||
if (version === null) {
|
||||
return reply.code(400).send({
|
||||
message: "image version must use v{number} format",
|
||||
statusCode: 400,
|
||||
})
|
||||
}
|
||||
|
||||
const width = parseOptionalInteger(request.query.w)
|
||||
const quality = parseOptionalInteger(request.query.q)
|
||||
const format = request.query.f ?? "auto"
|
||||
|
||||
return reply.code(501).header("cache-control", "no-store").send({
|
||||
assetId: request.params.assetId,
|
||||
format,
|
||||
message: "image gateway read-through pipeline is not implemented yet",
|
||||
preset: request.params.preset,
|
||||
quality,
|
||||
status: "not_implemented",
|
||||
version,
|
||||
width,
|
||||
})
|
||||
},
|
||||
)
|
||||
|
||||
app.all("/api/*", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||
app.all("/docs", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||
app.all("/docs/*", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||
app.all("/docs-json", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||
|
||||
app.setNotFoundHandler(async (_request, reply) => {
|
||||
return reply.code(404).send({
|
||||
message: "route not found",
|
||||
statusCode: 404,
|
||||
})
|
||||
})
|
||||
|
||||
return app
|
||||
}
|
||||
|
||||
type ImageQuery = {
|
||||
f?: string
|
||||
q?: string
|
||||
w?: string
|
||||
}
|
||||
|
||||
type ImageParams = {
|
||||
assetId: string
|
||||
preset: string
|
||||
version: string
|
||||
}
|
||||
|
||||
function parseVersionParam(value: string) {
|
||||
if (!value.startsWith("v")) {
|
||||
return null
|
||||
}
|
||||
|
||||
const parsed = Number.parseInt(value.slice(1), 10)
|
||||
|
||||
return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : null
|
||||
}
|
||||
|
||||
function parseOptionalInteger(value: string | undefined) {
|
||||
if (!value) {
|
||||
return null
|
||||
}
|
||||
|
||||
const parsed = Number.parseInt(value, 10)
|
||||
|
||||
return Number.isFinite(parsed) ? parsed : null
|
||||
}
|
||||
6
apps/gateway/tsconfig.build.json
Normal file
6
apps/gateway/tsconfig.build.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false
|
||||
}
|
||||
}
|
||||
22
apps/gateway/tsconfig.json
Normal file
22
apps/gateway/tsconfig.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"lib": ["ES2023"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"resolveJsonModule": true,
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["dist", "node_modules"]
|
||||
}
|
||||
24
apps/worker/package.json
Normal file
24
apps/worker/package.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "@image-platform/worker",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"dev": "tsx watch src/main.ts",
|
||||
"start": "node dist/main.js",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"@image-platform/database": "workspace:*",
|
||||
"@image-platform/queue": "workspace:*",
|
||||
"@image-platform/storage": "workspace:*",
|
||||
"amqplib": "^1.0.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/amqplib": "^0.10.8",
|
||||
"@types/node": "^25.6.0",
|
||||
"tsx": "^4.21.0",
|
||||
"typescript": "^6.0.3"
|
||||
}
|
||||
}
|
||||
25
apps/worker/src/config.ts
Normal file
25
apps/worker/src/config.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
import { loadQueueTopologyFromEnv, type QueueTopology } from "@image-platform/queue"
|
||||
|
||||
export type WorkerConfig = {
|
||||
prefetch: number
|
||||
queueTopology: QueueTopology
|
||||
rabbitmqUrl: string
|
||||
}
|
||||
|
||||
export function loadWorkerConfig(env: NodeJS.ProcessEnv = process.env): WorkerConfig {
|
||||
return {
|
||||
prefetch: parsePositiveInteger(env.WORKER_PREFETCH, 2),
|
||||
queueTopology: loadQueueTopologyFromEnv(env),
|
||||
rabbitmqUrl: env.RABBITMQ_URL ?? "amqp://image:image-password@localhost:5672/image_platform",
|
||||
}
|
||||
}
|
||||
|
||||
function parsePositiveInteger(value: string | undefined, fallback: number) {
|
||||
if (!value) {
|
||||
return fallback
|
||||
}
|
||||
|
||||
const parsed = Number.parseInt(value, 10)
|
||||
|
||||
return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : fallback
|
||||
}
|
||||
67
apps/worker/src/main.ts
Normal file
67
apps/worker/src/main.ts
Normal file
@@ -0,0 +1,67 @@
|
||||
import amqp, { type Channel, type ConsumeMessage } from "amqplib"
|
||||
import { parseGenerateVariantJobBuffer, type QueueTopology } from "@image-platform/queue"
|
||||
|
||||
import { loadWorkerConfig } from "./config.js"
|
||||
|
||||
async function bootstrap() {
|
||||
const config = loadWorkerConfig()
|
||||
const connection = await amqp.connect(config.rabbitmqUrl)
|
||||
const channel = await connection.createChannel()
|
||||
|
||||
await assertQueueTopology(channel, config.queueTopology)
|
||||
await channel.prefetch(config.prefetch)
|
||||
await channel.consume(
|
||||
config.queueTopology.generateVariantQueue,
|
||||
(message) => void handleGenerateVariantMessage(channel, message),
|
||||
{ noAck: false },
|
||||
)
|
||||
|
||||
console.log(`worker consuming ${config.queueTopology.generateVariantQueue}`)
|
||||
|
||||
const shutdown = async () => {
|
||||
console.log("worker shutting down")
|
||||
await channel.close().catch((error: unknown) => console.error("failed to close RabbitMQ channel", error))
|
||||
await connection.close().catch((error: unknown) => console.error("failed to close RabbitMQ connection", error))
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
process.once("SIGINT", () => void shutdown())
|
||||
process.once("SIGTERM", () => void shutdown())
|
||||
}
|
||||
|
||||
async function assertQueueTopology(channel: Channel, topology: QueueTopology) {
|
||||
await channel.assertExchange(topology.jobsExchange, "direct", { durable: true })
|
||||
await channel.assertExchange(topology.jobsDeadLetterExchange, "direct", { durable: true })
|
||||
await channel.assertQueue(topology.generateVariantQueue, {
|
||||
deadLetterExchange: topology.jobsDeadLetterExchange,
|
||||
deadLetterRoutingKey: topology.generateVariantDeadLetterRoutingKey,
|
||||
durable: true,
|
||||
})
|
||||
await channel.assertQueue(topology.generateVariantDeadLetterQueue, { durable: true })
|
||||
await channel.bindQueue(topology.generateVariantQueue, topology.jobsExchange, topology.generateVariantRoutingKey)
|
||||
await channel.bindQueue(
|
||||
topology.generateVariantDeadLetterQueue,
|
||||
topology.jobsDeadLetterExchange,
|
||||
topology.generateVariantDeadLetterRoutingKey,
|
||||
)
|
||||
}
|
||||
|
||||
async function handleGenerateVariantMessage(channel: Channel, message: ConsumeMessage | null) {
|
||||
if (message === null) {
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
const job = parseGenerateVariantJobBuffer(message.content)
|
||||
console.log("generate variant job received, handler not implemented yet", job)
|
||||
channel.nack(message, false, false)
|
||||
} catch (error) {
|
||||
console.error("invalid generate variant job", error)
|
||||
channel.nack(message, false, false)
|
||||
}
|
||||
}
|
||||
|
||||
void bootstrap().catch((error: unknown) => {
|
||||
console.error("worker failed to start", error)
|
||||
process.exit(1)
|
||||
})
|
||||
7
apps/worker/tsconfig.build.json
Normal file
7
apps/worker/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false
|
||||
},
|
||||
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||
}
|
||||
21
apps/worker/tsconfig.json
Normal file
21
apps/worker/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"lib": ["ES2023"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["dist", "node_modules"]
|
||||
}
|
||||
@@ -1,124 +0,0 @@
|
||||
# Черновик API Контракта
|
||||
|
||||
Это не реализация API, а фиксация будущего контракта для NestJS backend.
|
||||
|
||||
Backend отдаёт JSON, metadata, statuses и URLs. Он не должен проксировать image bytes на каждый обычный запрос.
|
||||
|
||||
## Allowed Hosts
|
||||
|
||||
```text
|
||||
GET /allowed-hosts
|
||||
POST /allowed-hosts
|
||||
PATCH /allowed-hosts/:id
|
||||
DELETE /allowed-hosts/:id
|
||||
```
|
||||
|
||||
## Assets
|
||||
|
||||
```text
|
||||
GET /assets
|
||||
POST /assets
|
||||
GET /assets/:id
|
||||
DELETE /assets/:id
|
||||
```
|
||||
|
||||
`POST /assets` request:
|
||||
|
||||
```json
|
||||
{
|
||||
"sourceUrl": "https://example.com/photo.jpg"
|
||||
}
|
||||
```
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- validate source URL;
|
||||
- check `allowed_image_hosts`;
|
||||
- create or reuse `image_assets` row;
|
||||
- optionally save original to S3 later.
|
||||
|
||||
## Variants
|
||||
|
||||
```text
|
||||
GET /assets/:id/variants
|
||||
POST /assets/:id/variants
|
||||
POST /variants/:id/regenerate
|
||||
DELETE /variants/:id
|
||||
```
|
||||
|
||||
`POST /assets/:id/variants` request:
|
||||
|
||||
```json
|
||||
{
|
||||
"preset": "card",
|
||||
"format": "webp",
|
||||
"width": 640
|
||||
}
|
||||
```
|
||||
|
||||
Response if ready:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "variant_123",
|
||||
"status": "ready",
|
||||
"url": "http://localhost:8888/images/asset_123/w640_q80_card.webp"
|
||||
}
|
||||
```
|
||||
|
||||
Response if generation is async:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "variant_123",
|
||||
"status": "pending",
|
||||
"url": null
|
||||
}
|
||||
```
|
||||
|
||||
## Image URLs For UI
|
||||
|
||||
Для UI нужен endpoint, который возвращает готовый набор URLs для `<picture>`/`srcset`.
|
||||
|
||||
```text
|
||||
GET /assets/:id/picture?preset=card
|
||||
```
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"assetId": "asset_123",
|
||||
"preset": "card",
|
||||
"sources": [
|
||||
{
|
||||
"type": "image/avif",
|
||||
"srcset": "http://localhost:8888/images/asset_123/w320_card.avif 320w, http://localhost:8888/images/asset_123/w640_card.avif 640w"
|
||||
},
|
||||
{
|
||||
"type": "image/webp",
|
||||
"srcset": "http://localhost:8888/images/asset_123/w320_card.webp 320w, http://localhost:8888/images/asset_123/w640_card.webp 640w"
|
||||
}
|
||||
],
|
||||
"fallback": {
|
||||
"src": "http://localhost:8888/images/asset_123/w640_card.jpg",
|
||||
"width": 640,
|
||||
"height": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Worker Lifecycle
|
||||
|
||||
Первый MVP может генерировать sync на request. Если генерация тяжёлая, variant создаётся как `pending`, а worker обрабатывает job.
|
||||
|
||||
PostgreSQL может выступить первой очередью:
|
||||
|
||||
```text
|
||||
SELECT * FROM image_variants
|
||||
WHERE status = 'pending'
|
||||
FOR UPDATE SKIP LOCKED
|
||||
LIMIT 1
|
||||
```
|
||||
|
||||
Позже можно добавить Redis/Valkey или отдельную queue, если PostgreSQL станет узким местом.
|
||||
@@ -12,24 +12,49 @@
|
||||
|---|---|---|
|
||||
| PostgreSQL | сейчас | Источник правды для assets, variants, hosts, statuses |
|
||||
| S3/MinIO | сейчас | Хранилище originals и generated variants |
|
||||
| API | позже | JSON API, admin operations, validation, orchestration |
|
||||
| Worker | позже | Генерация variants, upload в S3, update PostgreSQL |
|
||||
| Admin UI | позже | Управление hosts/assets/variants/presets |
|
||||
| Gateway | позже | Caddy/Souin hot cache и delivery layer |
|
||||
| Backend | сейчас | NestJS JSON API, Swagger, PostgreSQL/S3/RabbitMQ orchestration |
|
||||
| Worker | позже | RabbitMQ consumer, imgproxy processing, upload в S3, update PostgreSQL |
|
||||
| Admin UI | сейчас | React/Vite UI для будущего управления hosts/assets/variants/presets |
|
||||
| Gateway | сейчас | Fastify public image origin, L1 memory cache, root routing, без DB/S3 доступа |
|
||||
| RabbitMQ | сейчас | Очередь задач генерации variants |
|
||||
| imgproxy | external | CPU-heavy image processing |
|
||||
|
||||
## Архитектурное решение
|
||||
|
||||
Нужное поведение - Cloudinary-like: публичный URL изображения сам запускает read-through pipeline, если variant ещё не создан.
|
||||
|
||||
`image-platform` строится ради совместимости с `next/image` как custom loader provider. Next.js application не должен заранее вызывать API, ждать генерацию и затем подставлять S3 URL. Он должен передавать `src`, `width` и `quality` в loader, а loader должен вернуть стабильный URL нашего image origin.
|
||||
|
||||
Gateway поэтому является обязательной частью public delivery path, а не опциональным кешем.
|
||||
|
||||
## Целевая delivery схема
|
||||
|
||||
```text
|
||||
client
|
||||
-> CDN optional
|
||||
-> gateway Caddy/Souin
|
||||
-> S3 ready variant
|
||||
-> generator fallback
|
||||
-> Fastify gateway L1 memory cache
|
||||
-> NestJS backend
|
||||
-> PostgreSQL + S3 ready variant
|
||||
-> RabbitMQ -> worker
|
||||
-> external imgproxy
|
||||
-> source image
|
||||
```
|
||||
|
||||
Read-through flow:
|
||||
|
||||
```text
|
||||
1. client запрашивает /images/{assetId}/v{version}/{preset}?w=640&q=80&f=auto
|
||||
2. CDN HIT -> ответ сразу
|
||||
3. Gateway L1 HIT -> ответ сразу
|
||||
4. Gateway L1 MISS -> Gateway вызывает Backend internal ensure endpoint
|
||||
5. Backend проверяет PostgreSQL/S3
|
||||
6. S3 HIT -> Backend стримит bytes Gateway, Gateway кладёт ответ в L1
|
||||
7. S3 MISS -> Backend ставит RabbitMQ job
|
||||
8. Worker вызывает external imgproxy и сохраняет результат в S3
|
||||
9. Worker обновляет PostgreSQL, Backend отдаёт готовые bytes Gateway
|
||||
10. Gateway кладёт ответ в L1 и отдаёт клиенту
|
||||
```
|
||||
|
||||
## Разделение ответственности
|
||||
|
||||
PostgreSQL отвечает на вопросы:
|
||||
@@ -49,29 +74,45 @@ S3 хранит байты:
|
||||
|
||||
Gateway отдаёт картинки:
|
||||
|
||||
- hot cache HIT - сразу из Souin;
|
||||
- cache MISS - из S3;
|
||||
- S3 MISS - через generator fallback.
|
||||
- L1 memory HIT - сразу из памяти;
|
||||
- L1 memory MISS - вызывает Backend;
|
||||
- не имеет доступа к PostgreSQL, S3 и RabbitMQ.
|
||||
|
||||
Backend не должен проксировать картинки на каждый обычный запрос. Он отдаёт JSON, статусы и URLs.
|
||||
Backend управляет процессами: PostgreSQL, S3 read, RabbitMQ enqueue, ожидание ready variant. Worker выполняет CPU-heavy generation через external imgproxy и пишет результат в S3.
|
||||
|
||||
## URL модель
|
||||
|
||||
Публичные URL должны быть стабильными и не раскрывать source URL:
|
||||
Публичные URL должны быть стабильными и не раскрывать source URL. Для Next/image provider основной URL должен принимать width/quality из loader:
|
||||
|
||||
```text
|
||||
/images/{assetId}/{variantHash}.{format}
|
||||
/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||
```
|
||||
|
||||
Примеры:
|
||||
|
||||
```text
|
||||
/images/asset_123/w640_q80_cfill.avif
|
||||
/images/asset_123/w640_q80_cfill.webp
|
||||
/images/asset_123/w640_q80_cfill.jpg
|
||||
/images/asset_123/v4/card?w=640&q=80&f=auto
|
||||
/images/asset_123/v4/hero?w=1920&q=80&f=auto
|
||||
```
|
||||
|
||||
Формат лучше делать явным в URL и отдавать через `<picture>`/`srcset`, а не выбирать по `Accept` header. Так CDN/S3/Gateway cache остаётся предсказуемым.
|
||||
`v{version}` берётся из `image_assets.current_version` и меняется при обновлении source image. Это даёт immutable cache без purge старых CDN/L1/S3 keys.
|
||||
|
||||
`f=auto` нужен для совместимости с `next/image` custom loader: Next передаёт в loader `src`, `width` и `quality`, но не выбирает AVIF/WebP сам при custom loader. Image origin должен выбрать формат по `Accept` header, как Cloudinary `f_auto`.
|
||||
|
||||
Из-за `f=auto` обязательно:
|
||||
|
||||
- S3 key должен включать фактически выбранный формат;
|
||||
- response должен выставлять `Vary: Accept`;
|
||||
- CDN и Gateway L1 cache должны учитывать `Accept`;
|
||||
- response должен выставлять `Cache-Control: public, max-age=31536000, immutable` для versioned assets.
|
||||
|
||||
Для ручного `<picture>`/`srcset` можно добавить явный формат позже:
|
||||
|
||||
```text
|
||||
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=avif
|
||||
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=webp
|
||||
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=jpg
|
||||
```
|
||||
|
||||
## External imgproxy
|
||||
|
||||
|
||||
224
docs/backend-contract-draft.md
Normal file
224
docs/backend-contract-draft.md
Normal file
@@ -0,0 +1,224 @@
|
||||
# Черновик Backend Контракта
|
||||
|
||||
Это черновик будущего бизнес-контракта для NestJS backend. Сейчас реализованы system endpoints, Swagger и placeholder для internal image ensure.
|
||||
|
||||
## System
|
||||
|
||||
```text
|
||||
GET /api/health
|
||||
GET /docs
|
||||
GET /docs-json
|
||||
```
|
||||
|
||||
NestJS backend отдаёт JSON, metadata, statuses и URLs. Public image origin находится в Fastify Gateway. Backend владеет PostgreSQL, S3 orchestration и RabbitMQ jobs.
|
||||
|
||||
## Internal Image Ensure
|
||||
|
||||
Этот internal endpoint вызывается Gateway на L1 miss. Gateway не ходит в DB/S3 напрямую.
|
||||
|
||||
```text
|
||||
POST /api/internal/images/ensure
|
||||
```
|
||||
|
||||
Текущий статус реализации - endpoint зарезервирован и возвращает `501 Not Implemented`. Gateway `/images/*` пока тоже возвращает placeholder `501 Not Implemented`.
|
||||
|
||||
Request body:
|
||||
|
||||
```json
|
||||
{
|
||||
"assetId": "asset_123",
|
||||
"version": 4,
|
||||
"preset": "card",
|
||||
"width": 640,
|
||||
"quality": 80,
|
||||
"format": "webp"
|
||||
}
|
||||
```
|
||||
|
||||
Query params:
|
||||
|
||||
| Param | Описание |
|
||||
|---|---|
|
||||
| `w` | целевая ширина, должна быть разрешена preset или округлена до ближайшей разрешённой |
|
||||
| `q` | качество, должно быть из allowlist качества |
|
||||
| `f` | `auto`, `avif`, `webp`, `jpg`; для Next/image по умолчанию `auto` |
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- проверить `assetId` и `preset`;
|
||||
- вычислить deterministic `variantHash`;
|
||||
- проверить PostgreSQL и S3;
|
||||
- если variant готов в S3, вернуть bytes или stream metadata для Gateway;
|
||||
- если variant отсутствует, создать/переиспользовать variant row;
|
||||
- поставить `generate-variant` job в RabbitMQ;
|
||||
- дождаться `ready` до timeout, чтобы первый `next/image` request мог получить картинку;
|
||||
- вернуть image response или metadata для Gateway.
|
||||
|
||||
Response headers:
|
||||
|
||||
```http
|
||||
Content-Type: image/avif | image/webp | image/jpeg
|
||||
Cache-Control: public, max-age=31536000, immutable
|
||||
Vary: Accept
|
||||
ETag: "..."
|
||||
```
|
||||
|
||||
Ошибки:
|
||||
|
||||
| Status | Когда |
|
||||
|---|---|
|
||||
| `400` | некорректные query params |
|
||||
| `404` | asset или preset не найден |
|
||||
| `409` | variant уже генерируется и sync ожидание отключено |
|
||||
| `422` | source image нельзя обработать |
|
||||
| `502` | external imgproxy недоступен |
|
||||
|
||||
Public endpoint реализуется в Fastify Gateway, internal ensure endpoint - в Backend:
|
||||
|
||||
```text
|
||||
client -> CDN -> Gateway /images/* -> Backend ensure -> RabbitMQ -> Worker -> imgproxy -> S3
|
||||
```
|
||||
|
||||
Важно: `/images/*` не должен жить в NestJS `/api`. Gateway остаётся public image origin, Backend остаётся управлением процессов.
|
||||
Public image URL versioned: `/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto`.
|
||||
|
||||
## Allowed Hosts
|
||||
|
||||
```text
|
||||
GET /allowed-hosts
|
||||
POST /allowed-hosts
|
||||
PATCH /allowed-hosts/:id
|
||||
DELETE /allowed-hosts/:id
|
||||
```
|
||||
|
||||
## Assets
|
||||
|
||||
```text
|
||||
GET /assets
|
||||
POST /assets
|
||||
GET /assets/:id
|
||||
DELETE /assets/:id
|
||||
```
|
||||
|
||||
`POST /assets` request:
|
||||
|
||||
```json
|
||||
{
|
||||
"sourceUrl": "https://example.com/photo.jpg"
|
||||
}
|
||||
```
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- validate source URL;
|
||||
- check `allowed_image_hosts`;
|
||||
- create or reuse `image_assets` row;
|
||||
- optionally save original to S3 later.
|
||||
|
||||
## Variants
|
||||
|
||||
```text
|
||||
GET /assets/:id/variants
|
||||
POST /assets/:id/variants
|
||||
POST /variants/:id/regenerate
|
||||
DELETE /variants/:id
|
||||
```
|
||||
|
||||
`POST /assets/:id/variants` request:
|
||||
|
||||
```json
|
||||
{
|
||||
"preset": "card",
|
||||
"format": "webp",
|
||||
"width": 640
|
||||
}
|
||||
```
|
||||
|
||||
Response if ready:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "variant_123",
|
||||
"status": "ready",
|
||||
"url": "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=webp"
|
||||
}
|
||||
```
|
||||
|
||||
Response if generation is async:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "variant_123",
|
||||
"status": "pending",
|
||||
"url": null
|
||||
}
|
||||
```
|
||||
|
||||
## Image URLs For UI
|
||||
|
||||
Для ручного UI можно добавить endpoint, который возвращает готовый набор URLs для `<picture>`/`srcset`. Для `next/image` основным контрактом остаётся custom loader из `docs/next-image-provider.md`.
|
||||
|
||||
```text
|
||||
GET /assets/:id/picture?preset=card
|
||||
```
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"assetId": "asset_123",
|
||||
"preset": "card",
|
||||
"sources": [
|
||||
{
|
||||
"type": "image/avif",
|
||||
"srcset": "http://localhost:8888/images/asset_123/v4/card?w=320&q=80&f=avif 320w, http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=avif 640w"
|
||||
},
|
||||
{
|
||||
"type": "image/webp",
|
||||
"srcset": "http://localhost:8888/images/asset_123/v4/card?w=320&q=80&f=webp 320w, http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=webp 640w"
|
||||
}
|
||||
],
|
||||
"fallback": {
|
||||
"src": "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=jpg",
|
||||
"width": 640,
|
||||
"height": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Worker Lifecycle
|
||||
|
||||
Worker выполняет задачи из RabbitMQ. Задачи создаёт Backend.
|
||||
|
||||
RabbitMQ topology:
|
||||
|
||||
```text
|
||||
exchange: image-platform.jobs
|
||||
queue: image.generate-variant
|
||||
dead-letter exchange: image-platform.jobs.dlx
|
||||
dead-letter queue: image.generate-variant.dlq
|
||||
```
|
||||
|
||||
Job payload должен быть минимальным:
|
||||
|
||||
```json
|
||||
{
|
||||
"jobId": "job_123",
|
||||
"variantId": "variant_123"
|
||||
}
|
||||
```
|
||||
|
||||
Worker читает детали variant из PostgreSQL, вызывает imgproxy, пишет результат в S3 и обновляет status в PostgreSQL.
|
||||
|
||||
Если генерация тяжёлая или не успела завершиться до timeout, Backend может вернуть Gateway `504`, а job продолжит выполняться/retry по очереди.
|
||||
|
||||
PostgreSQL может выступить первой очередью:
|
||||
|
||||
```text
|
||||
SELECT * FROM image_variants
|
||||
WHERE status = 'pending'
|
||||
FOR UPDATE SKIP LOCKED
|
||||
LIMIT 1
|
||||
```
|
||||
|
||||
PostgreSQL-backed очередь не используется как основной механизм: для jobs выбран RabbitMQ.
|
||||
@@ -1,16 +1,16 @@
|
||||
# Черновик Data Model
|
||||
# Data Model
|
||||
|
||||
Это черновик для будущих миграций PostgreSQL. Реальные таблицы добавим вместе с API.
|
||||
Текущая PostgreSQL модель описана в `packages/database/src/schema.ts` и миграциях Drizzle в `packages/database/drizzle`.
|
||||
|
||||
## allowed_image_hosts
|
||||
|
||||
```text
|
||||
id
|
||||
hostname
|
||||
enabled
|
||||
description nullable
|
||||
created_at
|
||||
updated_at
|
||||
id uuid pk default gen_random_uuid()
|
||||
hostname text not null unique
|
||||
enabled boolean not null default true
|
||||
description text nullable
|
||||
created_at timestamptz not null default now()
|
||||
updated_at timestamptz not null default now()
|
||||
```
|
||||
|
||||
Правила normalization:
|
||||
@@ -26,53 +26,119 @@ updated_at
|
||||
## image_assets
|
||||
|
||||
```text
|
||||
id
|
||||
source_url
|
||||
source_host
|
||||
source_hash
|
||||
original_s3_key nullable
|
||||
status
|
||||
width nullable
|
||||
height nullable
|
||||
content_type nullable
|
||||
size_bytes nullable
|
||||
created_at
|
||||
updated_at
|
||||
id uuid pk default gen_random_uuid()
|
||||
public_id text not null unique
|
||||
current_version integer not null default 1
|
||||
status asset_status not null default active
|
||||
created_at timestamptz not null default now()
|
||||
updated_at timestamptz not null default now()
|
||||
```
|
||||
|
||||
`public_id` - стабильный идентификатор в public URL. `current_version` указывает активную версию source image и используется для cache invalidation без purge.
|
||||
|
||||
## image_asset_versions
|
||||
|
||||
```text
|
||||
id uuid pk default gen_random_uuid()
|
||||
asset_id uuid not null references image_assets(id) on delete cascade
|
||||
version integer not null
|
||||
source_url text not null
|
||||
source_host text not null
|
||||
source_hash text not null
|
||||
original_s3_key text nullable
|
||||
width integer nullable
|
||||
height integer nullable
|
||||
content_type text nullable
|
||||
size_bytes bigint nullable
|
||||
created_at timestamptz not null default now()
|
||||
```
|
||||
|
||||
Каждое изменение source image создаёт новую версию. Старые versioned URLs остаются immutable, новые клиенты получают URL с новым `v{version}`.
|
||||
|
||||
## image_variants
|
||||
|
||||
```text
|
||||
id
|
||||
asset_id
|
||||
preset
|
||||
variant_hash
|
||||
format
|
||||
width
|
||||
height nullable
|
||||
quality
|
||||
s3_key
|
||||
status: pending | processing | ready | failed
|
||||
size_bytes nullable
|
||||
error nullable
|
||||
created_at
|
||||
updated_at
|
||||
last_accessed_at nullable
|
||||
id uuid pk default gen_random_uuid()
|
||||
asset_id uuid not null references image_assets(id) on delete cascade
|
||||
asset_version_id uuid not null references image_asset_versions(id) on delete cascade
|
||||
asset_version integer not null
|
||||
preset text not null
|
||||
variant_hash text not null unique
|
||||
requested_format requested_format not null default auto
|
||||
format variant_format not null
|
||||
width integer not null
|
||||
height integer nullable
|
||||
quality integer not null
|
||||
s3_key text not null unique
|
||||
content_type text nullable
|
||||
etag text nullable
|
||||
status variant_status not null default pending
|
||||
size_bytes bigint nullable
|
||||
error text nullable
|
||||
attempt_count integer not null default 0
|
||||
last_accessed_at timestamptz nullable
|
||||
created_at timestamptz not null default now()
|
||||
updated_at timestamptz not null default now()
|
||||
```
|
||||
|
||||
`requested_format` хранит то, что запросил клиент (`auto`, `avif`, `webp`, `jpg`, `png`). `format` хранит фактический output format после negotiation по `Accept`.
|
||||
|
||||
## Enums
|
||||
|
||||
```text
|
||||
asset_status: active | disabled | deleted
|
||||
requested_format: auto | avif | webp | jpg | png
|
||||
variant_format: avif | webp | jpg | png
|
||||
variant_status: pending | processing | ready | failed
|
||||
```
|
||||
|
||||
## Unique constraints
|
||||
|
||||
```text
|
||||
allowed_image_hosts(hostname)
|
||||
image_assets(source_hash)
|
||||
image_variants(asset_id, variant_hash, format)
|
||||
image_assets(public_id)
|
||||
image_asset_versions(asset_id, version)
|
||||
image_variants(asset_id, asset_version, preset, width, quality, format)
|
||||
image_variants(s3_key)
|
||||
image_variants(variant_hash)
|
||||
```
|
||||
|
||||
Индексы:
|
||||
|
||||
```text
|
||||
image_asset_versions(source_hash)
|
||||
image_variants(status)
|
||||
```
|
||||
|
||||
## S3 layout
|
||||
|
||||
```text
|
||||
originals/{assetId}/source
|
||||
variants/{assetId}/{variantHash}.{format}
|
||||
originals/{assetId}/v{version}/source
|
||||
variants/{assetId}/v{version}/{variantHash}.{format}
|
||||
```
|
||||
|
||||
`variantHash` должен включать:
|
||||
|
||||
- `assetId`;
|
||||
- `assetVersion`;
|
||||
- `preset`;
|
||||
- normalized width;
|
||||
- normalized quality;
|
||||
- фактический output format;
|
||||
- параметры transform, влияющие на bytes.
|
||||
|
||||
Для `f=auto` в public URL в S3 всё равно пишется фактический формат:
|
||||
|
||||
```text
|
||||
variants/asset_123/v4/card_w640_q80_avif.avif
|
||||
variants/asset_123/v4/card_w640_q80_webp.webp
|
||||
variants/asset_123/v4/card_w640_q80_jpg.jpg
|
||||
```
|
||||
|
||||
Public URL также versioned:
|
||||
|
||||
```text
|
||||
/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||
```
|
||||
|
||||
## Presets
|
||||
|
||||
@@ -2,15 +2,18 @@
|
||||
|
||||
## Принцип
|
||||
|
||||
В Docker запускаем стабильную инфраструктуру. Кодовые сервисы позже запускаем нодой с hot reload.
|
||||
В Docker запускаем стабильную инфраструктуру. Кодовые сервисы запускаем нодой с hot reload.
|
||||
|
||||
Сейчас в Docker есть только:
|
||||
|
||||
- PostgreSQL;
|
||||
- MinIO;
|
||||
- MinIO bucket init.
|
||||
- MinIO bucket init;
|
||||
- imgproxy dev instance;
|
||||
- RabbitMQ.
|
||||
|
||||
`api`, `worker`, `admin` и `gateway` пока не созданы.
|
||||
`backend` уже создан как NestJS app. `admin` уже создан как чистый Vite React TS app. `gateway` уже создан как Fastify app. `worker` уже создан как RabbitMQ skeleton.
|
||||
Gateway обязателен для Cloudinary-like поведения и интеграции с `next/image`.
|
||||
|
||||
## Запуск инфраструктуры
|
||||
|
||||
@@ -45,40 +48,139 @@ pnpm infra:logs
|
||||
| PostgreSQL | `localhost:5433` |
|
||||
| MinIO API | `http://localhost:9000` |
|
||||
| MinIO Console | `http://localhost:9001` |
|
||||
| imgproxy | `http://localhost:18080` |
|
||||
| RabbitMQ | `amqp://localhost:5672` |
|
||||
| RabbitMQ Management | `http://localhost:15672` |
|
||||
| Backend API | `http://localhost:3001/api` |
|
||||
| Swagger | `http://localhost:3001/docs` |
|
||||
| Admin | `http://localhost:5173` |
|
||||
| Gateway | `http://localhost:8888` |
|
||||
|
||||
Если `localhost:8888` занят старым `image-gateway`, остановите старый stack или задайте `GATEWAY_PORT=8890` в `.env`.
|
||||
|
||||
## Backend
|
||||
|
||||
Запустить NestJS backend:
|
||||
|
||||
```bash
|
||||
pnpm backend:dev
|
||||
```
|
||||
|
||||
Проверки:
|
||||
|
||||
```bash
|
||||
curl http://localhost:3001/api/health
|
||||
open http://localhost:3001/docs
|
||||
```
|
||||
|
||||
## Database
|
||||
|
||||
Drizzle schema живёт в `packages/database/src/schema.ts`, миграции - в `packages/database/drizzle`.
|
||||
|
||||
Сгенерировать миграцию после изменения schema:
|
||||
|
||||
```bash
|
||||
pnpm db:generate
|
||||
```
|
||||
|
||||
Применить миграции к локальному PostgreSQL:
|
||||
|
||||
```bash
|
||||
pnpm db:migrate
|
||||
```
|
||||
|
||||
Открыть Drizzle Studio из корня проекта:
|
||||
|
||||
```bash
|
||||
pnpm db:studio
|
||||
```
|
||||
|
||||
Проверить database package:
|
||||
|
||||
```bash
|
||||
pnpm db:typecheck
|
||||
pnpm db:build
|
||||
```
|
||||
|
||||
## Admin
|
||||
|
||||
Запустить React/Vite admin:
|
||||
|
||||
```bash
|
||||
pnpm admin:dev
|
||||
```
|
||||
|
||||
Открыть:
|
||||
|
||||
```bash
|
||||
open http://localhost:5173
|
||||
```
|
||||
|
||||
## Gateway
|
||||
|
||||
Gateway запускается нодой:
|
||||
|
||||
```bash
|
||||
pnpm gateway:dev
|
||||
```
|
||||
|
||||
Проверить gateway health:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8888/health
|
||||
```
|
||||
|
||||
Проверить placeholder image origin route:
|
||||
|
||||
```bash
|
||||
curl -i "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=auto"
|
||||
```
|
||||
|
||||
Ожидаемый текущий статус - `501`, потому что S3/imgproxy генерация ещё не реализована.
|
||||
|
||||
## Worker
|
||||
|
||||
Worker запускается нодой и сейчас только объявляет RabbitMQ topology, слушает `image.generate-variant` и отправляет полученные jobs в DLQ, потому что генерация ещё не реализована.
|
||||
|
||||
```bash
|
||||
pnpm worker:dev
|
||||
```
|
||||
|
||||
Проверить worker package без запуска consumer:
|
||||
|
||||
```bash
|
||||
pnpm worker:typecheck
|
||||
pnpm worker:build
|
||||
```
|
||||
|
||||
## Будущий dev flow
|
||||
|
||||
Когда появятся приложения:
|
||||
Текущая и будущая схема:
|
||||
|
||||
```text
|
||||
React/Vite admin localhost:5173
|
||||
-> NestJS API localhost:3001
|
||||
-> NestJS backend localhost:3001
|
||||
-> PostgreSQL localhost:5433
|
||||
-> MinIO localhost:9000
|
||||
-> RabbitMQ localhost:5672
|
||||
|
||||
worker node process
|
||||
-> PostgreSQL
|
||||
-> MinIO
|
||||
-> external imgproxy
|
||||
-> RabbitMQ
|
||||
-> imgproxy localhost:18080
|
||||
|
||||
gateway Caddy/Souin localhost:8888
|
||||
-> S3/MinIO ready variant
|
||||
-> API/generator fallback on host.docker.internal:3001
|
||||
Fastify gateway localhost:8888
|
||||
-> L1 memory cache
|
||||
-> Backend internal ensure endpoint
|
||||
```
|
||||
|
||||
Для Linux gateway container должен видеть host services через:
|
||||
## imgproxy для разработки
|
||||
|
||||
```yaml
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
```
|
||||
|
||||
## External imgproxy для разработки
|
||||
|
||||
`imgproxy` не входит в `image-platform` stack. Для локальной разработки можно использовать любой внешний endpoint и прописать его в `.env`:
|
||||
В dev compose поднимается локальный `imgproxy`, опубликованный только на `127.0.0.1:18080`:
|
||||
|
||||
```env
|
||||
IMGPROXY_UPSTREAM=http://localhost:18080
|
||||
```
|
||||
|
||||
Если нужен локальный standalone imgproxy, его можно запустить отдельно вне этого compose stack. Он остаётся внешней зависимостью платформы.
|
||||
Для production `imgproxy` всё равно рассматривается как внешняя зависимость и может жить на отдельном мощном сервере.
|
||||
|
||||
@@ -63,7 +63,7 @@ Final signed URL:
|
||||
## Security rules
|
||||
|
||||
- Не отдавать `IMGPROXY_KEY` и `IMGPROXY_SALT` в браузер.
|
||||
- Source URL валидировать в API/worker.
|
||||
- Source URL валидировать в Backend/worker.
|
||||
- Разрешать только `http` и `https`.
|
||||
- Запрещать localhost, private IP, loopback, link-local.
|
||||
- Source host должен быть enabled в `allowed_image_hosts`.
|
||||
|
||||
105
docs/next-image-provider.md
Normal file
105
docs/next-image-provider.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Next/image Provider
|
||||
|
||||
`image-platform` должен работать как custom image provider для `next/image`.
|
||||
|
||||
## Next.js contract
|
||||
|
||||
Next.js custom loader получает только:
|
||||
|
||||
- `src`;
|
||||
- `width`;
|
||||
- `quality`.
|
||||
|
||||
Поэтому public image URL должен принимать эти параметры напрямую и сам запускать read-through генерацию при miss.
|
||||
|
||||
## Loader config
|
||||
|
||||
В Next.js приложении используется `loaderFile`:
|
||||
|
||||
```js
|
||||
// next.config.js
|
||||
module.exports = {
|
||||
images: {
|
||||
loader: "custom",
|
||||
loaderFile: "./src/image-platform-loader.js",
|
||||
qualities: [60, 75, 80, 90],
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Пример loader:
|
||||
|
||||
```js
|
||||
"use client"
|
||||
|
||||
const imageBaseUrl = process.env.NEXT_PUBLIC_IMAGE_PLATFORM_URL
|
||||
|
||||
export default function imagePlatformLoader({ src, width, quality }) {
|
||||
const normalizedSrc = src.startsWith("/") ? src.slice(1) : src
|
||||
const q = quality || 80
|
||||
|
||||
return `${imageBaseUrl}/images/${normalizedSrc}?w=${width}&q=${q}&f=auto`
|
||||
}
|
||||
```
|
||||
|
||||
Пример использования:
|
||||
|
||||
```tsx
|
||||
import Image from "next/image"
|
||||
|
||||
export function ProductCard() {
|
||||
return <Image src="asset_123/v4/card" width={640} height={420} alt="Product" />
|
||||
}
|
||||
```
|
||||
|
||||
## Public URL
|
||||
|
||||
```text
|
||||
GET /images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||
```
|
||||
|
||||
Сейчас route уже зарезервирован в Fastify Gateway, но возвращает placeholder `501`, пока не реализованы PostgreSQL/S3/imgproxy read-through операции.
|
||||
|
||||
Пример:
|
||||
|
||||
```text
|
||||
https://img.example.com/images/asset_123/v4/card?w=640&q=80&f=auto
|
||||
```
|
||||
|
||||
`src` не должен быть source URL. Это должен быть versioned platform identifier, например `asset_123/v4/card`. Source URL хранится в PostgreSQL и не раскрывается в public image URL.
|
||||
|
||||
`v{version}` меняется при обновлении source image. Старые URL можно кэшировать как immutable без purge.
|
||||
|
||||
## Format auto
|
||||
|
||||
`f=auto` выбирает output format по `Accept` header:
|
||||
|
||||
1. `image/avif`, если клиент поддерживает AVIF и preset разрешает AVIF.
|
||||
2. `image/webp`, если клиент поддерживает WebP и preset разрешает WebP.
|
||||
3. `image/jpeg` или original fallback.
|
||||
|
||||
Для auto format обязательны headers:
|
||||
|
||||
```http
|
||||
Vary: Accept
|
||||
Cache-Control: public, max-age=31536000, immutable
|
||||
Content-Type: image/avif | image/webp | image/jpeg
|
||||
```
|
||||
|
||||
CDN и Gateway L1 cache должны учитывать `Vary: Accept`, иначе можно отдать AVIF клиенту без AVIF support.
|
||||
|
||||
## Read-through behavior
|
||||
|
||||
```text
|
||||
client -> CDN -> Fastify gateway -> L1 memory -> Backend -> RabbitMQ -> Worker -> imgproxy -> S3
|
||||
```
|
||||
|
||||
Поведение:
|
||||
|
||||
- CDN HIT: backend не вызывается.
|
||||
- Gateway L1 HIT: backend не вызывается.
|
||||
- Gateway L1 MISS: Gateway вызывает Backend internal ensure endpoint.
|
||||
- S3 HIT: Backend отдаёт bytes Gateway, Gateway кладёт result в L1.
|
||||
- S3 MISS: Backend ставит RabbitMQ job, Worker генерирует variant через external imgproxy, сохраняет в S3, обновляет PostgreSQL, Backend возвращает bytes Gateway.
|
||||
|
||||
Так достигается Cloudinary-like поведение: первый запрос создаёт derived asset, следующие запросы отдаются из cache/storage.
|
||||
@@ -46,6 +46,33 @@ services:
|
||||
mc anonymous set download local/$${S3_BUCKET}"
|
||||
restart: "no"
|
||||
|
||||
imgproxy:
|
||||
image: darthsim/imgproxy:latest
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:${IMGPROXY_PORT:-18080}:8080"
|
||||
environment:
|
||||
GODEBUG: http2client=0
|
||||
IMGPROXY_KEY: ${IMGPROXY_KEY:-}
|
||||
IMGPROXY_SALT: ${IMGPROXY_SALT:-}
|
||||
IMGPROXY_WORKERS: ${IMGPROXY_WORKERS:-2}
|
||||
IMGPROXY_MAX_SRC_RESOLUTION: ${IMGPROXY_MAX_SRC_RESOLUTION:-20}
|
||||
IMGPROXY_USE_ETAG: "true"
|
||||
IMGPROXY_ENABLE_VIDEO_THUMBNAILS: "false"
|
||||
IMGPROXY_DOWNLOAD_TIMEOUT: ${IMGPROXY_DOWNLOAD_TIMEOUT:-30}
|
||||
IMGPROXY_ALLOWED_SOURCES: ${IMGPROXY_ALLOWED_SOURCES:-}
|
||||
|
||||
rabbitmq:
|
||||
image: rabbitmq:4-management-alpine
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER:-image}
|
||||
RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS:-image-password}
|
||||
RABBITMQ_DEFAULT_VHOST: ${RABBITMQ_DEFAULT_VHOST:-image_platform}
|
||||
ports:
|
||||
- "127.0.0.1:${RABBITMQ_PORT:-5672}:5672"
|
||||
- "127.0.0.1:${RABBITMQ_MANAGEMENT_PORT:-15672}:15672"
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
minio-data:
|
||||
|
||||
27
package.json
27
package.json
@@ -9,10 +9,35 @@
|
||||
"pnpm": ">=10.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"admin:build": "pnpm --filter @image-platform/admin build",
|
||||
"admin:dev": "pnpm --filter @image-platform/admin dev",
|
||||
"admin:preview": "pnpm --filter @image-platform/admin preview",
|
||||
"admin:typecheck": "pnpm --filter @image-platform/admin typecheck",
|
||||
"backend:build": "pnpm --filter @image-platform/backend build",
|
||||
"backend:dev": "pnpm --filter @image-platform/backend dev",
|
||||
"backend:start": "pnpm --filter @image-platform/backend start",
|
||||
"backend:typecheck": "pnpm --filter @image-platform/backend typecheck",
|
||||
"db:build": "pnpm --filter @image-platform/database build",
|
||||
"db:generate": "pnpm --filter @image-platform/database db:generate",
|
||||
"db:migrate": "pnpm --filter @image-platform/database db:migrate",
|
||||
"db:studio": "pnpm --filter @image-platform/database db:studio",
|
||||
"db:typecheck": "pnpm --filter @image-platform/database typecheck",
|
||||
"gateway:build": "pnpm --filter @image-platform/gateway build",
|
||||
"gateway:dev": "pnpm --filter @image-platform/gateway dev",
|
||||
"gateway:start": "pnpm --filter @image-platform/gateway start",
|
||||
"gateway:typecheck": "pnpm --filter @image-platform/gateway typecheck",
|
||||
"queue:build": "pnpm --filter @image-platform/queue build",
|
||||
"queue:typecheck": "pnpm --filter @image-platform/queue typecheck",
|
||||
"storage:build": "pnpm --filter @image-platform/storage build",
|
||||
"storage:typecheck": "pnpm --filter @image-platform/storage typecheck",
|
||||
"worker:build": "pnpm db:build && pnpm queue:build && pnpm storage:build && pnpm --filter @image-platform/worker build",
|
||||
"worker:dev": "pnpm db:build && pnpm queue:build && pnpm storage:build && pnpm --filter @image-platform/worker dev",
|
||||
"worker:start": "pnpm --filter @image-platform/worker start",
|
||||
"worker:typecheck": "pnpm --filter @image-platform/worker typecheck",
|
||||
"infra:config": "docker compose -f infra/compose.dev.yml config",
|
||||
"infra:up": "docker compose -f infra/compose.dev.yml up -d",
|
||||
"infra:down": "docker compose -f infra/compose.dev.yml down",
|
||||
"infra:logs": "docker compose -f infra/compose.dev.yml logs -f",
|
||||
"check": "pnpm infra:config"
|
||||
"check": "pnpm infra:config && pnpm db:typecheck && pnpm queue:typecheck && pnpm storage:typecheck && pnpm backend:typecheck && pnpm admin:typecheck && pnpm gateway:typecheck && pnpm worker:typecheck"
|
||||
}
|
||||
}
|
||||
|
||||
12
packages/database/drizzle.config.ts
Normal file
12
packages/database/drizzle.config.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
import { defineConfig } from "drizzle-kit"
|
||||
|
||||
export default defineConfig({
|
||||
dbCredentials: {
|
||||
url: process.env.DATABASE_URL ?? "postgres://image:image-password@localhost:5433/image_platform",
|
||||
},
|
||||
dialect: "postgresql",
|
||||
out: "./drizzle",
|
||||
schema: "./src/schema.ts",
|
||||
strict: true,
|
||||
verbose: true,
|
||||
})
|
||||
72
packages/database/drizzle/0000_calm_magdalene.sql
Normal file
72
packages/database/drizzle/0000_calm_magdalene.sql
Normal file
@@ -0,0 +1,72 @@
|
||||
CREATE TYPE "public"."asset_status" AS ENUM('active', 'disabled', 'deleted');--> statement-breakpoint
|
||||
CREATE TYPE "public"."requested_format" AS ENUM('auto', 'avif', 'webp', 'jpg', 'png');--> statement-breakpoint
|
||||
CREATE TYPE "public"."variant_format" AS ENUM('avif', 'webp', 'jpg', 'png');--> statement-breakpoint
|
||||
CREATE TYPE "public"."variant_status" AS ENUM('pending', 'processing', 'ready', 'failed');--> statement-breakpoint
|
||||
CREATE TABLE "allowed_image_hosts" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"hostname" text NOT NULL,
|
||||
"enabled" boolean DEFAULT true NOT NULL,
|
||||
"description" text,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "image_asset_versions" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"asset_id" uuid NOT NULL,
|
||||
"version" integer NOT NULL,
|
||||
"source_url" text NOT NULL,
|
||||
"source_host" text NOT NULL,
|
||||
"source_hash" text NOT NULL,
|
||||
"original_s3_key" text,
|
||||
"width" integer,
|
||||
"height" integer,
|
||||
"content_type" text,
|
||||
"size_bytes" bigint,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "image_assets" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"public_id" text NOT NULL,
|
||||
"current_version" integer DEFAULT 1 NOT NULL,
|
||||
"status" "asset_status" DEFAULT 'active' NOT NULL,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "image_variants" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"asset_id" uuid NOT NULL,
|
||||
"asset_version_id" uuid NOT NULL,
|
||||
"asset_version" integer NOT NULL,
|
||||
"preset" text NOT NULL,
|
||||
"variant_hash" text NOT NULL,
|
||||
"requested_format" "requested_format" DEFAULT 'auto' NOT NULL,
|
||||
"format" "variant_format" NOT NULL,
|
||||
"width" integer NOT NULL,
|
||||
"height" integer,
|
||||
"quality" integer NOT NULL,
|
||||
"s3_key" text NOT NULL,
|
||||
"content_type" text,
|
||||
"etag" text,
|
||||
"status" "variant_status" DEFAULT 'pending' NOT NULL,
|
||||
"size_bytes" bigint,
|
||||
"error" text,
|
||||
"attempt_count" integer DEFAULT 0 NOT NULL,
|
||||
"last_accessed_at" timestamp with time zone,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "image_asset_versions" ADD CONSTRAINT "image_asset_versions_asset_id_image_assets_id_fk" FOREIGN KEY ("asset_id") REFERENCES "public"."image_assets"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||
ALTER TABLE "image_variants" ADD CONSTRAINT "image_variants_asset_id_image_assets_id_fk" FOREIGN KEY ("asset_id") REFERENCES "public"."image_assets"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||
ALTER TABLE "image_variants" ADD CONSTRAINT "image_variants_asset_version_id_image_asset_versions_id_fk" FOREIGN KEY ("asset_version_id") REFERENCES "public"."image_asset_versions"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "allowed_image_hosts_hostname_idx" ON "allowed_image_hosts" USING btree ("hostname");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "image_asset_versions_asset_version_idx" ON "image_asset_versions" USING btree ("asset_id","version");--> statement-breakpoint
|
||||
CREATE INDEX "image_asset_versions_source_hash_idx" ON "image_asset_versions" USING btree ("source_hash");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "image_assets_public_id_idx" ON "image_assets" USING btree ("public_id");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "image_variants_lookup_idx" ON "image_variants" USING btree ("asset_id","asset_version","preset","width","quality","format");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "image_variants_s3_key_idx" ON "image_variants" USING btree ("s3_key");--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "image_variants_variant_hash_idx" ON "image_variants" USING btree ("variant_hash");--> statement-breakpoint
|
||||
CREATE INDEX "image_variants_status_idx" ON "image_variants" USING btree ("status");
|
||||
604
packages/database/drizzle/meta/0000_snapshot.json
Normal file
604
packages/database/drizzle/meta/0000_snapshot.json
Normal file
@@ -0,0 +1,604 @@
|
||||
{
|
||||
"id": "72292622-d326-46fe-8e6a-90096c7e6634",
|
||||
"prevId": "00000000-0000-0000-0000-000000000000",
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"tables": {
|
||||
"public.allowed_image_hosts": {
|
||||
"name": "allowed_image_hosts",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"hostname": {
|
||||
"name": "hostname",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"enabled": {
|
||||
"name": "enabled",
|
||||
"type": "boolean",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": true
|
||||
},
|
||||
"description": {
|
||||
"name": "description",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"allowed_image_hosts_hostname_idx": {
|
||||
"name": "allowed_image_hosts_hostname_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "hostname",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.image_asset_versions": {
|
||||
"name": "image_asset_versions",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"asset_id": {
|
||||
"name": "asset_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"version": {
|
||||
"name": "version",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"source_url": {
|
||||
"name": "source_url",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"source_host": {
|
||||
"name": "source_host",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"source_hash": {
|
||||
"name": "source_hash",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"original_s3_key": {
|
||||
"name": "original_s3_key",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"width": {
|
||||
"name": "width",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"height": {
|
||||
"name": "height",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"content_type": {
|
||||
"name": "content_type",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"size_bytes": {
|
||||
"name": "size_bytes",
|
||||
"type": "bigint",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"image_asset_versions_asset_version_idx": {
|
||||
"name": "image_asset_versions_asset_version_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "asset_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "version",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"image_asset_versions_source_hash_idx": {
|
||||
"name": "image_asset_versions_source_hash_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "source_hash",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
"image_asset_versions_asset_id_image_assets_id_fk": {
|
||||
"name": "image_asset_versions_asset_id_image_assets_id_fk",
|
||||
"tableFrom": "image_asset_versions",
|
||||
"tableTo": "image_assets",
|
||||
"columnsFrom": [
|
||||
"asset_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
}
|
||||
},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.image_assets": {
|
||||
"name": "image_assets",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"public_id": {
|
||||
"name": "public_id",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"current_version": {
|
||||
"name": "current_version",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": 1
|
||||
},
|
||||
"status": {
|
||||
"name": "status",
|
||||
"type": "asset_status",
|
||||
"typeSchema": "public",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'active'"
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"image_assets_public_id_idx": {
|
||||
"name": "image_assets_public_id_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "public_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.image_variants": {
|
||||
"name": "image_variants",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"asset_id": {
|
||||
"name": "asset_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"asset_version_id": {
|
||||
"name": "asset_version_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"asset_version": {
|
||||
"name": "asset_version",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"preset": {
|
||||
"name": "preset",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"variant_hash": {
|
||||
"name": "variant_hash",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"requested_format": {
|
||||
"name": "requested_format",
|
||||
"type": "requested_format",
|
||||
"typeSchema": "public",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'auto'"
|
||||
},
|
||||
"format": {
|
||||
"name": "format",
|
||||
"type": "variant_format",
|
||||
"typeSchema": "public",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"width": {
|
||||
"name": "width",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"height": {
|
||||
"name": "height",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"quality": {
|
||||
"name": "quality",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"s3_key": {
|
||||
"name": "s3_key",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"content_type": {
|
||||
"name": "content_type",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"etag": {
|
||||
"name": "etag",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"status": {
|
||||
"name": "status",
|
||||
"type": "variant_status",
|
||||
"typeSchema": "public",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'pending'"
|
||||
},
|
||||
"size_bytes": {
|
||||
"name": "size_bytes",
|
||||
"type": "bigint",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"error": {
|
||||
"name": "error",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"attempt_count": {
|
||||
"name": "attempt_count",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": 0
|
||||
},
|
||||
"last_accessed_at": {
|
||||
"name": "last_accessed_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"image_variants_lookup_idx": {
|
||||
"name": "image_variants_lookup_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "asset_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "asset_version",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "preset",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "width",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "quality",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "format",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"image_variants_s3_key_idx": {
|
||||
"name": "image_variants_s3_key_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "s3_key",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"image_variants_variant_hash_idx": {
|
||||
"name": "image_variants_variant_hash_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "variant_hash",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"image_variants_status_idx": {
|
||||
"name": "image_variants_status_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "status",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
"image_variants_asset_id_image_assets_id_fk": {
|
||||
"name": "image_variants_asset_id_image_assets_id_fk",
|
||||
"tableFrom": "image_variants",
|
||||
"tableTo": "image_assets",
|
||||
"columnsFrom": [
|
||||
"asset_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
},
|
||||
"image_variants_asset_version_id_image_asset_versions_id_fk": {
|
||||
"name": "image_variants_asset_version_id_image_asset_versions_id_fk",
|
||||
"tableFrom": "image_variants",
|
||||
"tableTo": "image_asset_versions",
|
||||
"columnsFrom": [
|
||||
"asset_version_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
}
|
||||
},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
}
|
||||
},
|
||||
"enums": {
|
||||
"public.asset_status": {
|
||||
"name": "asset_status",
|
||||
"schema": "public",
|
||||
"values": [
|
||||
"active",
|
||||
"disabled",
|
||||
"deleted"
|
||||
]
|
||||
},
|
||||
"public.requested_format": {
|
||||
"name": "requested_format",
|
||||
"schema": "public",
|
||||
"values": [
|
||||
"auto",
|
||||
"avif",
|
||||
"webp",
|
||||
"jpg",
|
||||
"png"
|
||||
]
|
||||
},
|
||||
"public.variant_format": {
|
||||
"name": "variant_format",
|
||||
"schema": "public",
|
||||
"values": [
|
||||
"avif",
|
||||
"webp",
|
||||
"jpg",
|
||||
"png"
|
||||
]
|
||||
},
|
||||
"public.variant_status": {
|
||||
"name": "variant_status",
|
||||
"schema": "public",
|
||||
"values": [
|
||||
"pending",
|
||||
"processing",
|
||||
"ready",
|
||||
"failed"
|
||||
]
|
||||
}
|
||||
},
|
||||
"schemas": {},
|
||||
"sequences": {},
|
||||
"roles": {},
|
||||
"policies": {},
|
||||
"views": {},
|
||||
"_meta": {
|
||||
"columns": {},
|
||||
"schemas": {},
|
||||
"tables": {}
|
||||
}
|
||||
}
|
||||
13
packages/database/drizzle/meta/_journal.json
Normal file
13
packages/database/drizzle/meta/_journal.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"entries": [
|
||||
{
|
||||
"idx": 0,
|
||||
"version": "7",
|
||||
"when": 1777963363578,
|
||||
"tag": "0000_calm_magdalene",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
30
packages/database/package.json
Normal file
30
packages/database/package.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"name": "@image-platform/database",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./src/index.ts",
|
||||
"default": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"types": "./src/index.ts",
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"db:generate": "drizzle-kit generate --config drizzle.config.ts",
|
||||
"db:migrate": "drizzle-kit migrate --config drizzle.config.ts",
|
||||
"db:studio": "drizzle-kit studio --config drizzle.config.ts",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"pg": "^8.20.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.6.0",
|
||||
"@types/pg": "^8.20.0",
|
||||
"drizzle-kit": "^0.31.10",
|
||||
"typescript": "^6.0.3"
|
||||
}
|
||||
}
|
||||
21
packages/database/src/client.ts
Normal file
21
packages/database/src/client.ts
Normal file
@@ -0,0 +1,21 @@
|
||||
import { drizzle } from "drizzle-orm/node-postgres"
|
||||
import pg from "pg"
|
||||
|
||||
import * as schema from "./schema.js"
|
||||
|
||||
const { Pool } = pg
|
||||
|
||||
export type DatabasePool = pg.Pool
|
||||
export type Database = ReturnType<typeof createDatabase>
|
||||
|
||||
export function createDatabasePool(databaseUrl = process.env.DATABASE_URL) {
|
||||
if (!databaseUrl) {
|
||||
throw new Error("DATABASE_URL is required")
|
||||
}
|
||||
|
||||
return new Pool({ connectionString: databaseUrl })
|
||||
}
|
||||
|
||||
export function createDatabase(pool: DatabasePool) {
|
||||
return drizzle(pool, { schema })
|
||||
}
|
||||
2
packages/database/src/index.ts
Normal file
2
packages/database/src/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export { createDatabase, createDatabasePool } from "./client.js"
|
||||
export * from "./schema.js"
|
||||
113
packages/database/src/schema.ts
Normal file
113
packages/database/src/schema.ts
Normal file
@@ -0,0 +1,113 @@
|
||||
import {
|
||||
bigint,
|
||||
boolean,
|
||||
index,
|
||||
integer,
|
||||
pgEnum,
|
||||
pgTable,
|
||||
text,
|
||||
timestamp,
|
||||
uniqueIndex,
|
||||
uuid,
|
||||
} from "drizzle-orm/pg-core"
|
||||
|
||||
export const assetStatusEnum = pgEnum("asset_status", ["active", "disabled", "deleted"])
|
||||
export const variantFormatEnum = pgEnum("variant_format", ["avif", "webp", "jpg", "png"])
|
||||
export const requestedFormatEnum = pgEnum("requested_format", ["auto", "avif", "webp", "jpg", "png"])
|
||||
export const variantStatusEnum = pgEnum("variant_status", ["pending", "processing", "ready", "failed"])
|
||||
|
||||
const timestamps = {
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}
|
||||
|
||||
export const allowedImageHosts = pgTable(
|
||||
"allowed_image_hosts",
|
||||
{
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
hostname: text("hostname").notNull(),
|
||||
enabled: boolean("enabled").notNull().default(true),
|
||||
description: text("description"),
|
||||
...timestamps,
|
||||
},
|
||||
(table) => [uniqueIndex("allowed_image_hosts_hostname_idx").on(table.hostname)],
|
||||
)
|
||||
|
||||
export const imageAssets = pgTable(
|
||||
"image_assets",
|
||||
{
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
publicId: text("public_id").notNull(),
|
||||
currentVersion: integer("current_version").notNull().default(1),
|
||||
status: assetStatusEnum("status").notNull().default("active"),
|
||||
...timestamps,
|
||||
},
|
||||
(table) => [uniqueIndex("image_assets_public_id_idx").on(table.publicId)],
|
||||
)
|
||||
|
||||
export const imageAssetVersions = pgTable(
|
||||
"image_asset_versions",
|
||||
{
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
assetId: uuid("asset_id")
|
||||
.notNull()
|
||||
.references(() => imageAssets.id, { onDelete: "cascade" }),
|
||||
version: integer("version").notNull(),
|
||||
sourceUrl: text("source_url").notNull(),
|
||||
sourceHost: text("source_host").notNull(),
|
||||
sourceHash: text("source_hash").notNull(),
|
||||
originalS3Key: text("original_s3_key"),
|
||||
width: integer("width"),
|
||||
height: integer("height"),
|
||||
contentType: text("content_type"),
|
||||
sizeBytes: bigint("size_bytes", { mode: "number" }),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
},
|
||||
(table) => [
|
||||
uniqueIndex("image_asset_versions_asset_version_idx").on(table.assetId, table.version),
|
||||
index("image_asset_versions_source_hash_idx").on(table.sourceHash),
|
||||
],
|
||||
)
|
||||
|
||||
export const imageVariants = pgTable(
|
||||
"image_variants",
|
||||
{
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
assetId: uuid("asset_id")
|
||||
.notNull()
|
||||
.references(() => imageAssets.id, { onDelete: "cascade" }),
|
||||
assetVersionId: uuid("asset_version_id")
|
||||
.notNull()
|
||||
.references(() => imageAssetVersions.id, { onDelete: "cascade" }),
|
||||
assetVersion: integer("asset_version").notNull(),
|
||||
preset: text("preset").notNull(),
|
||||
variantHash: text("variant_hash").notNull(),
|
||||
requestedFormat: requestedFormatEnum("requested_format").notNull().default("auto"),
|
||||
format: variantFormatEnum("format").notNull(),
|
||||
width: integer("width").notNull(),
|
||||
height: integer("height"),
|
||||
quality: integer("quality").notNull(),
|
||||
s3Key: text("s3_key").notNull(),
|
||||
contentType: text("content_type"),
|
||||
etag: text("etag"),
|
||||
status: variantStatusEnum("status").notNull().default("pending"),
|
||||
sizeBytes: bigint("size_bytes", { mode: "number" }),
|
||||
error: text("error"),
|
||||
attemptCount: integer("attempt_count").notNull().default(0),
|
||||
lastAccessedAt: timestamp("last_accessed_at", { withTimezone: true }),
|
||||
...timestamps,
|
||||
},
|
||||
(table) => [
|
||||
uniqueIndex("image_variants_lookup_idx").on(
|
||||
table.assetId,
|
||||
table.assetVersion,
|
||||
table.preset,
|
||||
table.width,
|
||||
table.quality,
|
||||
table.format,
|
||||
),
|
||||
uniqueIndex("image_variants_s3_key_idx").on(table.s3Key),
|
||||
uniqueIndex("image_variants_variant_hash_idx").on(table.variantHash),
|
||||
index("image_variants_status_idx").on(table.status),
|
||||
],
|
||||
)
|
||||
7
packages/database/tsconfig.build.json
Normal file
7
packages/database/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false
|
||||
},
|
||||
"exclude": ["dist", "node_modules", "drizzle.config.ts"]
|
||||
}
|
||||
21
packages/database/tsconfig.json
Normal file
21
packages/database/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"lib": ["ES2023"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["dist", "node_modules"]
|
||||
}
|
||||
21
packages/queue/package.json
Normal file
21
packages/queue/package.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"name": "@image-platform/queue",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./src/index.ts",
|
||||
"default": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"types": "./src/index.ts",
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.6.0",
|
||||
"typescript": "^6.0.3"
|
||||
}
|
||||
}
|
||||
2
packages/queue/src/index.ts
Normal file
2
packages/queue/src/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export * from "./jobs.js"
|
||||
export * from "./topology.js"
|
||||
37
packages/queue/src/jobs.ts
Normal file
37
packages/queue/src/jobs.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
export type GenerateVariantJob = {
|
||||
jobId: string
|
||||
variantId: string
|
||||
}
|
||||
|
||||
export function parseGenerateVariantJobBuffer(buffer: Buffer): GenerateVariantJob {
|
||||
const value = JSON.parse(buffer.toString("utf8")) as unknown
|
||||
|
||||
return parseGenerateVariantJob(value)
|
||||
}
|
||||
|
||||
export function parseGenerateVariantJob(value: unknown): GenerateVariantJob {
|
||||
if (!isRecord(value)) {
|
||||
throw new Error("generate variant job must be a JSON object")
|
||||
}
|
||||
|
||||
if (!isNonEmptyString(value.jobId)) {
|
||||
throw new Error("generate variant job must include jobId")
|
||||
}
|
||||
|
||||
if (!isNonEmptyString(value.variantId)) {
|
||||
throw new Error("generate variant job must include variantId")
|
||||
}
|
||||
|
||||
return {
|
||||
jobId: value.jobId,
|
||||
variantId: value.variantId,
|
||||
}
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value)
|
||||
}
|
||||
|
||||
function isNonEmptyString(value: unknown): value is string {
|
||||
return typeof value === "string" && value.trim().length > 0
|
||||
}
|
||||
33
packages/queue/src/topology.ts
Normal file
33
packages/queue/src/topology.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
export type QueueTopology = {
|
||||
generateVariantDeadLetterQueue: string
|
||||
generateVariantDeadLetterRoutingKey: string
|
||||
generateVariantQueue: string
|
||||
generateVariantRoutingKey: string
|
||||
jobsDeadLetterExchange: string
|
||||
jobsExchange: string
|
||||
}
|
||||
|
||||
export const DEFAULT_QUEUE_TOPOLOGY: QueueTopology = {
|
||||
generateVariantDeadLetterQueue: "image.generate-variant.dlq",
|
||||
generateVariantDeadLetterRoutingKey: "image.generate-variant.dlq",
|
||||
generateVariantQueue: "image.generate-variant",
|
||||
generateVariantRoutingKey: "image.generate-variant",
|
||||
jobsDeadLetterExchange: "image-platform.jobs.dlx",
|
||||
jobsExchange: "image-platform.jobs",
|
||||
}
|
||||
|
||||
export function loadQueueTopologyFromEnv(env: NodeJS.ProcessEnv = process.env): QueueTopology {
|
||||
const generateVariantQueue = env.RABBITMQ_GENERATE_VARIANT_QUEUE ?? DEFAULT_QUEUE_TOPOLOGY.generateVariantQueue
|
||||
const generateVariantDeadLetterQueue =
|
||||
env.RABBITMQ_GENERATE_VARIANT_DLQ ?? DEFAULT_QUEUE_TOPOLOGY.generateVariantDeadLetterQueue
|
||||
|
||||
return {
|
||||
generateVariantDeadLetterQueue,
|
||||
generateVariantDeadLetterRoutingKey:
|
||||
env.RABBITMQ_GENERATE_VARIANT_DLQ_ROUTING_KEY ?? generateVariantDeadLetterQueue,
|
||||
generateVariantQueue,
|
||||
generateVariantRoutingKey: env.RABBITMQ_GENERATE_VARIANT_ROUTING_KEY ?? generateVariantQueue,
|
||||
jobsDeadLetterExchange: env.RABBITMQ_GENERATE_VARIANT_DLX ?? DEFAULT_QUEUE_TOPOLOGY.jobsDeadLetterExchange,
|
||||
jobsExchange: env.RABBITMQ_JOBS_EXCHANGE ?? DEFAULT_QUEUE_TOPOLOGY.jobsExchange,
|
||||
}
|
||||
}
|
||||
7
packages/queue/tsconfig.build.json
Normal file
7
packages/queue/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false
|
||||
},
|
||||
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||
}
|
||||
21
packages/queue/tsconfig.json
Normal file
21
packages/queue/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"lib": ["ES2023"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["dist", "node_modules"]
|
||||
}
|
||||
24
packages/storage/package.json
Normal file
24
packages/storage/package.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "@image-platform/storage",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./src/index.ts",
|
||||
"default": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"types": "./src/index.ts",
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.1042.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.6.0",
|
||||
"typescript": "^6.0.3"
|
||||
}
|
||||
}
|
||||
20
packages/storage/src/client.ts
Normal file
20
packages/storage/src/client.ts
Normal file
@@ -0,0 +1,20 @@
|
||||
import { S3Client, type S3ClientConfig } from "@aws-sdk/client-s3"
|
||||
|
||||
import type { StorageConfig } from "./config.js"
|
||||
|
||||
export function createS3Client(config: StorageConfig) {
|
||||
const clientConfig: S3ClientConfig = {
|
||||
endpoint: config.endpoint,
|
||||
forcePathStyle: config.forcePathStyle,
|
||||
region: config.region,
|
||||
}
|
||||
|
||||
if (config.accessKeyId && config.secretAccessKey) {
|
||||
clientConfig.credentials = {
|
||||
accessKeyId: config.accessKeyId,
|
||||
secretAccessKey: config.secretAccessKey,
|
||||
}
|
||||
}
|
||||
|
||||
return new S3Client(clientConfig)
|
||||
}
|
||||
31
packages/storage/src/config.ts
Normal file
31
packages/storage/src/config.ts
Normal file
@@ -0,0 +1,31 @@
|
||||
export type StorageConfig = {
|
||||
accessKeyId?: string
|
||||
bucket: string
|
||||
endpoint?: string
|
||||
forcePathStyle: boolean
|
||||
region: string
|
||||
secretAccessKey?: string
|
||||
}
|
||||
|
||||
export function loadStorageConfigFromEnv(env: NodeJS.ProcessEnv = process.env): StorageConfig {
|
||||
return {
|
||||
accessKeyId: normalizeOptionalString(env.S3_ACCESS_KEY_ID),
|
||||
bucket: env.S3_BUCKET ?? "image-platform",
|
||||
endpoint: normalizeOptionalString(env.S3_ENDPOINT),
|
||||
forcePathStyle: parseBoolean(env.S3_FORCE_PATH_STYLE, true),
|
||||
region: env.S3_REGION ?? "us-east-1",
|
||||
secretAccessKey: normalizeOptionalString(env.S3_SECRET_ACCESS_KEY),
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeOptionalString(value: string | undefined) {
|
||||
return value && value.trim().length > 0 ? value : undefined
|
||||
}
|
||||
|
||||
function parseBoolean(value: string | undefined, fallback: boolean) {
|
||||
if (value === undefined) {
|
||||
return fallback
|
||||
}
|
||||
|
||||
return ["1", "true", "yes"].includes(value.toLowerCase())
|
||||
}
|
||||
3
packages/storage/src/index.ts
Normal file
3
packages/storage/src/index.ts
Normal file
@@ -0,0 +1,3 @@
|
||||
export * from "./client.js"
|
||||
export * from "./config.js"
|
||||
export * from "./keys.js"
|
||||
40
packages/storage/src/keys.ts
Normal file
40
packages/storage/src/keys.ts
Normal file
@@ -0,0 +1,40 @@
|
||||
export type VariantFormat = "avif" | "jpg" | "png" | "webp"
|
||||
|
||||
export type OriginalImageKeyInput = {
|
||||
assetId: string
|
||||
version: number
|
||||
}
|
||||
|
||||
export type VariantImageKeyInput = OriginalImageKeyInput & {
|
||||
format: VariantFormat
|
||||
variantHash: string
|
||||
}
|
||||
|
||||
export function buildOriginalImageKey(input: OriginalImageKeyInput) {
|
||||
return `originals/${safeSegment(input.assetId, "assetId")}/v${safeVersion(input.version)}/source`
|
||||
}
|
||||
|
||||
export function buildVariantImageKey(input: VariantImageKeyInput) {
|
||||
return [
|
||||
"variants",
|
||||
safeSegment(input.assetId, "assetId"),
|
||||
`v${safeVersion(input.version)}`,
|
||||
`${safeSegment(input.variantHash, "variantHash")}.${input.format}`,
|
||||
].join("/")
|
||||
}
|
||||
|
||||
function safeSegment(value: string, name: string) {
|
||||
if (value.length === 0 || value.includes("/")) {
|
||||
throw new Error(`${name} must be a non-empty S3 key segment`)
|
||||
}
|
||||
|
||||
return value
|
||||
}
|
||||
|
||||
function safeVersion(value: number) {
|
||||
if (!Number.isSafeInteger(value) || value < 1) {
|
||||
throw new Error("version must be a positive integer")
|
||||
}
|
||||
|
||||
return value
|
||||
}
|
||||
7
packages/storage/tsconfig.build.json
Normal file
7
packages/storage/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false
|
||||
},
|
||||
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||
}
|
||||
21
packages/storage/tsconfig.json
Normal file
21
packages/storage/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"declaration": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"lib": ["ES2023"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"skipLibCheck": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"target": "ES2023",
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["dist", "node_modules"]
|
||||
}
|
||||
5979
pnpm-lock.yaml
generated
5979
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user