Upgrade Readme
This commit is contained in:
361
docs/ARCHITECTURE_OVERVIEW.md
Normal file
361
docs/ARCHITECTURE_OVERVIEW.md
Normal file
@ -0,0 +1,361 @@
|
||||
# Architecture Overview
|
||||
|
||||
## Hybrid Architecture
|
||||
|
||||
Project Notes uses a **hybrid architecture** that combines the best of multiple paradigms:
|
||||
|
||||
### Core Components
|
||||
|
||||
- **Go Backend**: Fast, type-safe server handling file operations and indexing
|
||||
- **HTMX**: "HTML over the wire" for dynamic interactions with minimal JavaScript
|
||||
- **Modern JavaScript**: CodeMirror 6, drag-and-drop, and UI enhancements
|
||||
- **Vite**: Modern build tool for efficient JavaScript bundling
|
||||
|
||||
### Key Design Principles
|
||||
|
||||
1. **Server renders HTML, not JSON** (simpler, faster)
|
||||
2. **HTMX handles all AJAX and DOM updates** (consistent, reliable)
|
||||
3. **JavaScript enhances UI** (editor, drag-and-drop, animations)
|
||||
4. **Event-driven coordination** between HTMX and JavaScript
|
||||
|
||||
## Technology Stack
|
||||
|
||||
### Backend: Go
|
||||
|
||||
- **`net/http`**: Standard library for the web server
|
||||
- **`github.com/fsnotify/fsnotify`**: For watching file system changes and re-indexing
|
||||
- **`gopkg.in/yaml.v3`**: For parsing and marshaling YAML front matter
|
||||
- **Chi Router**: Lightweight, fast HTTP router (implied by usage)
|
||||
|
||||
**Why Go?**
|
||||
- Fast compilation and execution
|
||||
- Excellent standard library
|
||||
- Built-in concurrency
|
||||
- Single binary deployment
|
||||
- Cross-platform support (Linux, FreeBSD, macOS, Windows)
|
||||
|
||||
### Frontend: HTML, CSS, JavaScript
|
||||
|
||||
#### Core Technologies
|
||||
|
||||
- **[htmx](https://htmx.org/)**: For dynamic UI interactions without writing much JavaScript
|
||||
- Declarative AJAX requests
|
||||
- DOM swapping and updates
|
||||
- WebSocket support
|
||||
- Event-driven architecture
|
||||
|
||||
- **[CodeMirror 6](https://codemirror.net/6/)**: For the robust Markdown editor
|
||||
- Extensible architecture
|
||||
- Syntax highlighting
|
||||
- Vim mode support
|
||||
- Custom extensions (slash commands)
|
||||
|
||||
- **[Vite](https://vitejs.dev/)**: For bundling frontend JavaScript modules
|
||||
- Fast development server
|
||||
- Optimized production builds
|
||||
- ES modules support
|
||||
- Hot module replacement
|
||||
|
||||
#### Supporting Libraries
|
||||
|
||||
- **[marked.js](https://marked.js.org/)**: For client-side Markdown parsing in the preview
|
||||
- **[DOMPurify](https://dompurpurify.com/)**: For sanitizing HTML output from Markdown to prevent XSS vulnerabilities
|
||||
- **[Highlight.js](https://highlightjs.org/)**: For syntax highlighting in code blocks
|
||||
- **Custom CSS theme**: Dark mode inspired by VS Code and GitHub Dark
|
||||
|
||||
**Why This Stack?**
|
||||
- Minimal JavaScript complexity
|
||||
- Progressive enhancement
|
||||
- Fast page loads
|
||||
- SEO-friendly (server-rendered HTML)
|
||||
- Easy to understand and maintain
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### Server-Side Rendering (SSR)
|
||||
|
||||
All HTML is rendered on the server using Go's `html/template` package:
|
||||
- Initial page loads are fast
|
||||
- No JavaScript required for basic functionality
|
||||
- Better SEO and accessibility
|
||||
|
||||
### Progressive Enhancement
|
||||
|
||||
The application works without JavaScript but is enhanced with it:
|
||||
1. **Base functionality**: Browse notes, view content (no JS)
|
||||
2. **HTMX enhancement**: Dynamic updates without page reloads
|
||||
3. **JavaScript enhancement**: Rich editor, drag-and-drop, animations
|
||||
|
||||
### File-Based Storage
|
||||
|
||||
Notes are stored as plain Markdown files with YAML front matter:
|
||||
```markdown
|
||||
---
|
||||
title: My Note
|
||||
date: 2025-11-11
|
||||
last_modified: 2025-11-11:14:30
|
||||
tags:
|
||||
- example
|
||||
- markdown
|
||||
---
|
||||
|
||||
# My Note
|
||||
|
||||
Content here...
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- No database setup required
|
||||
- Easy backups (just copy files)
|
||||
- Version control friendly (Git)
|
||||
- Human-readable
|
||||
- Portable (works with any Markdown tool)
|
||||
|
||||
### In-Memory Indexing
|
||||
|
||||
Notes are indexed in memory for fast search:
|
||||
- Full-text search across title, tags, path, content
|
||||
- Tag-based filtering
|
||||
- Path-based navigation
|
||||
- Real-time updates via file system watcher
|
||||
|
||||
**Trade-offs**:
|
||||
- Memory usage scales with note count
|
||||
- Index rebuilt on server restart
|
||||
- Suitable for personal/small team use (< 10,000 notes)
|
||||
|
||||
## Request Flow
|
||||
|
||||
### Reading a Note
|
||||
|
||||
```
|
||||
Browser → GET /editor?note=path/to/note.md
|
||||
↓
|
||||
Go Handler
|
||||
↓
|
||||
Read file from disk
|
||||
↓
|
||||
Parse front matter
|
||||
↓
|
||||
Render HTML template
|
||||
↓
|
||||
Browser ← HTML response
|
||||
↓
|
||||
CodeMirror initializes
|
||||
↓
|
||||
User sees editable note
|
||||
```
|
||||
|
||||
### Saving a Note
|
||||
|
||||
```
|
||||
Browser → htmx POST /save
|
||||
↓
|
||||
Go Handler
|
||||
↓
|
||||
Update front matter (last_modified)
|
||||
↓
|
||||
Write file to disk
|
||||
↓
|
||||
File system watcher detects change
|
||||
↓
|
||||
Re-index note
|
||||
↓
|
||||
Browser ← Success response
|
||||
↓
|
||||
htmx updates UI
|
||||
```
|
||||
|
||||
### Search
|
||||
|
||||
```
|
||||
Browser → htmx GET /search?q=query
|
||||
↓
|
||||
Go Handler
|
||||
↓
|
||||
Query in-memory index
|
||||
↓
|
||||
Score and rank results
|
||||
↓
|
||||
Render search results template
|
||||
↓
|
||||
Browser ← HTML fragment
|
||||
↓
|
||||
htmx swaps into DOM
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Filesystem (notes/) ←→ File Watcher (fsnotify)
|
||||
↓
|
||||
Indexer (in-memory)
|
||||
↓
|
||||
HTTP Handlers
|
||||
↓
|
||||
Templates + HTMX
|
||||
↓
|
||||
Browser
|
||||
↓
|
||||
CodeMirror Editor
|
||||
```
|
||||
|
||||
## Scalability Considerations
|
||||
|
||||
### Current Design (Suitable for)
|
||||
|
||||
- Personal use: 1-10,000 notes
|
||||
- Small teams: 2-5 users
|
||||
- Single server deployment
|
||||
- Notes up to ~1MB each
|
||||
|
||||
### Limitations
|
||||
|
||||
- **No concurrent editing**: Last write wins
|
||||
- **In-memory index**: Limited by server RAM
|
||||
- **No authentication**: Requires reverse proxy
|
||||
- **Single server**: No horizontal scaling
|
||||
|
||||
### Future Enhancements (if needed)
|
||||
|
||||
- SQLite for metadata indexing (larger note collections)
|
||||
- WebSocket for real-time collaboration
|
||||
- JWT authentication built-in
|
||||
- Redis for distributed caching
|
||||
- Object storage for large attachments
|
||||
|
||||
## Security Model
|
||||
|
||||
### Current State
|
||||
|
||||
- **No built-in authentication**: Designed for local/private networks
|
||||
- **XSS protection**: DOMPurify sanitizes Markdown output
|
||||
- **Path traversal prevention**: Input validation on file paths
|
||||
- **CSRF**: Not needed (no session-based auth)
|
||||
|
||||
### Recommended Production Setup
|
||||
|
||||
```
|
||||
Internet → Reverse Proxy (nginx/Caddy)
|
||||
↓
|
||||
Basic Auth / OAuth
|
||||
↓
|
||||
Project Notes (Go)
|
||||
↓
|
||||
Filesystem (notes/)
|
||||
```
|
||||
|
||||
Example nginx config:
|
||||
```nginx
|
||||
location / {
|
||||
auth_basic "Project Notes";
|
||||
auth_basic_user_file /etc/nginx/.htpasswd;
|
||||
proxy_pass http://localhost:8080;
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Strengths
|
||||
|
||||
- **Fast page loads**: Server-rendered HTML
|
||||
- **Low latency**: In-memory indexing
|
||||
- **Efficient search**: Pre-indexed content
|
||||
- **Small footprint**: ~10-20MB RAM for typical usage
|
||||
|
||||
### Benchmarks (approximate)
|
||||
|
||||
- Note load time: < 50ms
|
||||
- Search query: < 10ms (1000 notes)
|
||||
- Save operation: < 100ms
|
||||
- Index rebuild: < 1s (1000 notes)
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Backend Development
|
||||
|
||||
```bash
|
||||
# Run with auto-reload (using air or similar)
|
||||
air
|
||||
|
||||
# Or manual reload
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
### Frontend Development
|
||||
|
||||
```bash
|
||||
# Watch mode (auto-rebuild)
|
||||
cd frontend
|
||||
npm run build -- --watch
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
go test ./...
|
||||
|
||||
# With coverage
|
||||
go test -cover ./...
|
||||
|
||||
# Specific package
|
||||
go test -v ./internal/indexer
|
||||
```
|
||||
|
||||
## Deployment Options
|
||||
|
||||
### 1. Simple Binary
|
||||
|
||||
```bash
|
||||
# Build
|
||||
go build -o server ./cmd/server
|
||||
|
||||
# Run
|
||||
./server -addr :8080 -notes-dir ~/notes
|
||||
```
|
||||
|
||||
### 2. Systemd Service (Linux)
|
||||
|
||||
See [FREEBSD_BUILD.md](./FREEBSD_BUILD.md) for service examples.
|
||||
|
||||
### 3. Docker (future)
|
||||
|
||||
```dockerfile
|
||||
FROM golang:1.22 AS builder
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN go build -o server ./cmd/server
|
||||
|
||||
FROM alpine:latest
|
||||
RUN apk add --no-cache ca-certificates
|
||||
COPY --from=builder /app/server /server
|
||||
COPY --from=builder /app/static /static
|
||||
COPY --from=builder /app/templates /templates
|
||||
EXPOSE 8080
|
||||
CMD ["/server"]
|
||||
```
|
||||
|
||||
### 4. Reverse Proxy
|
||||
|
||||
Always recommended for production:
|
||||
- nginx, Caddy, Traefik
|
||||
- TLS termination
|
||||
- Authentication
|
||||
- Rate limiting
|
||||
- Caching
|
||||
|
||||
## Documentation
|
||||
|
||||
For more detailed information, see:
|
||||
|
||||
- **[ARCHITECTURE.md](../ARCHITECTURE.md)** - Complete system architecture, patterns, and best practices
|
||||
- **[CLAUDE.md](../CLAUDE.md)** - Development guide and implementation details
|
||||
- **[API.md](../API.md)** - REST API documentation
|
||||
- **[DAILY_NOTES.md](./DAILY_NOTES.md)** - Daily notes feature guide
|
||||
- **[FREEBSD_BUILD.md](./FREEBSD_BUILD.md)** - FreeBSD deployment guide
|
||||
- **[CHANGELOG.md](../CHANGELOG.md)** - Version history
|
||||
|
||||
---
|
||||
|
||||
**Last updated**: November 11, 2025
|
||||
Reference in New Issue
Block a user