Commit graph

663 commits

Author SHA1 Message Date
Steven
4180613fc0 fix: update demo mode handling 2026-01-21 07:36:30 +08:00
Johnny
0f3c9a467d refactor: migrate HOST roles to ADMIN
- Updated the isSuperUser function to only check for ADMIN role.
- Added SQL migration scripts for MySQL, PostgreSQL, and SQLite to change user roles from HOST to ADMIN.
- Created a new SQLite migration to alter the user table structure and ensure data integrity during the migration process.
2026-01-20 23:38:30 +08:00
Johnny
47ebb04dc3 refactor: remove mode flag and introduce explicit demo flag 2026-01-20 22:58:33 +08:00
Johnny
05f31e457e fix: add mmap size setting to database connection to prevent OOM errors 2026-01-20 21:53:31 +08:00
Johnny
552318209b
fix: resolve flaky migration tests and add stable upgrade test (#5514) 2026-01-20 19:25:00 +08:00
Steven
00f21b86e2 chore: remove redundant tests 2026-01-20 09:12:36 +08:00
Johnny
7089db06c2 test: enhance memo filter tests with COALESCE for JSON extraction and add migration data persistence tests 2026-01-19 23:09:17 +08:00
Johnny
af2a2588bf chore(test): add edge case tests for user settings shortcuts and JSON fields 2026-01-19 22:50:14 +08:00
Johnny
dc7ec8a8ad feat: allow setting custom timestamps when creating memos and comments
Allow API users to set custom create_time, update_time, and display_time
when creating memos and comments. This enables importing historical data
with accurate timestamps.

Changes:
- Update proto definitions: change create_time and update_time from
  OUTPUT_ONLY to OPTIONAL to allow setting on creation
- Modify CreateMemo service to handle custom timestamps from request
- Update database drivers (SQLite, MySQL, PostgreSQL) to support
  inserting custom timestamps when provided
- Add comprehensive test coverage for custom timestamp functionality
- Maintain backward compatibility: auto-generated timestamps still
  work when custom values are not provided
- Fix golangci-lint issues in plugin/filter (godot and revive)

Fixes #5483
2026-01-17 12:56:03 +08:00
Johnny
cbf46a2988 feat(filter): add CEL list comprehension support for tag filtering
Add support for CEL exists() comprehension with startsWith, endsWith, and
contains predicates to enable powerful tag filtering patterns.

Features:
- tags.exists(t, t.startsWith("prefix")) - Match tags by prefix
- tags.exists(t, t.endsWith("suffix")) - Match tags by suffix
- tags.exists(t, t.contains("substring")) - Match tags by substring
- Negation: !tags.exists(...) to exclude matching tags
- Works with all operators (AND, OR, NOT) and other filters

Implementation:
- Added ListComprehensionCondition IR type for comprehension expressions
- Parser detects exists() macro and extracts predicates
- Renderer generates optimized SQL for SQLite, MySQL, PostgreSQL
- Proper NULL/empty array handling across all database dialects
- Helper functions reduce code duplication

Design decisions:
- Only exists() supported (all() rejected at parse time with clear error)
- Only simple predicates (matches() excluded to avoid regex complexity)
- Fail-fast validation with helpful error messages

Tests:
- Comprehensive test suite covering all predicates and edge cases
- Tests for NULL/empty arrays, combined filters, negation
- Real-world use case test for Issue #5480 (archive workflow)
- All tests pass on SQLite, MySQL, PostgreSQL

Closes #5480
2026-01-17 12:36:39 +08:00
Johnny
4bd0f29ee5 fix: resolve linter warning and cache race condition
- Suppress revive redundant-test-main-exit warning with //nolint
- Fix data race in cache.Clear() by using Delete() instead of map replacement
- Both changes maintain correct behavior while fixing CI failures
2026-01-14 23:50:37 +08:00
Johnny
db57f4456a fix: resolve data races in store tests with atomic operations
Fix all data races detected by -race flag in parallel store tests.

Problems fixed:
1. TestMain not propagating exit code
   - Was: m.Run(); return
   - Now: os.Exit(m.Run())

2. Data races on global DSN variables
   - mysqlBaseDSN and postgresBaseDSN written in sync.Once
   - Read outside Once without synchronization
   - Race detector: write/read without happens-before relationship

3. Data races on container pointers
   - mysqlContainer, postgresContainer, testDockerNetwork
   - Same pattern: write in Once, read in cleanup

Solution: Use atomic operations
- atomic.Value for DSN strings (Store/Load)
- atomic.Pointer for container pointers (Store/Load)
- Provides proper memory synchronization
- Race-free reads and writes

Why sync.Once alone wasn't enough:
- sync.Once guarantees function runs once
- Does NOT provide memory barrier for variables written inside
- Reads outside Once have no synchronization with writes
- Race detector correctly flags this as violation

Technical details:
- atomic.Value.Store() provides release semantics
- atomic.Value.Load() provides acquire semantics
- Guarantees happens-before relationship per Go memory model
- All 159 parallel tests can safely access globals

Impact:
- Tests now pass with -race flag
- No performance degradation (atomic ops are fast)
- Maintains parallel execution benefits (8-10x speedup)
- Proper Go memory model compliance

Related: Issues #2, #3 from race analysis
2026-01-14 23:37:12 +08:00
Johnny
411e8fc5b0 perf: enable parallel execution for all store tests
Add t.Parallel() to all 159 test functions in store/test/

Benefits:
- Tests run in parallel (8-16 concurrent on typical CI)
- Each test already has isolated database (safe to parallelize)
- No shared state between tests
- Go test runner handles synchronization

Expected performance:
- SQLite tests: 4-6min → 30-45sec (87% faster)
- MySQL tests: 6-8min → 45-60sec (88% faster)
- Better CPU utilization (10-15% → 80-95%)

Why it's safe:
- NewTestingStore() creates isolated DB per test
- No global state mutations
- Each test uses t.TempDir() for file isolation
- Container-based tests use unique database names

Impact on CI workflow:
- Backend tests workflow: 4-6min → 1-2min total
- Store test group: 3-4min → 20-30sec
- 8-10x speedup from parallelization alone
2026-01-14 22:44:19 +08:00
Johnny
c45a59549a fix: replace os.Exit with panic for clearer error handling 2026-01-12 23:36:48 +08:00
Johnny
69b62cccdb test: optimize store tests performance by reusing docker image and reducing build context 2026-01-12 23:30:56 +08:00
Johnny
f58533003b fix: clean up memo_relation and attachments when deleting memo
Fixes #5472

Move cleanup logic to store.DeleteMemo to ensure data consistency:
- Delete memo_relation records where memo is source (MemoID) or target (RelatedMemoID)
- Delete attachments linked to the memo (including S3/local files)

This prevents stale COMMENT records in memo_relation after deleting
a memo that has comments.
2026-01-12 23:18:04 +08:00
Johnny
6899c2f66a chore: fix golangci-lint 2026-01-12 22:32:38 +08:00
Johnny
31f634b71a chore: add more tests 2026-01-12 22:28:42 +08:00
Johnny
1e8505bfeb chore: fix tests 2026-01-08 23:06:28 +08:00
Johnny
af6a0726d8 chore: drop redundant indexes in sqlite migration 2026-01-08 22:53:46 +08:00
Johnny
1985205dc2 chore: remove memo_organizer table 2026-01-08 22:28:13 +08:00
Johnny
f4cf2e9559 test: improve migration tests stability and maintainability
- Fix linting issues and address testcontainers deprecation in store/test/containers.go
- Extract MemosStartupWaitStrategy for consistent container health checks
- Refactor migrator_test.go to use a consolidated testMigration helper, reducing duplication
- Add store/test/Dockerfile for optimized local test image builds
2026-01-07 23:04:12 +08:00
Johnny
14fb38f375 fix: attachment table name 2026-01-07 20:44:21 +08:00
Johnny
c29c1d3e1f fix: seed data 2026-01-07 20:41:16 +08:00
Johnny
cc9a214be8 revert: revert system_setting to instance_setting rename changes
Reverts only the system_setting → instance_setting rename related changes from commit d326c71.
Keeps the resource → attachment rename changes intact.

- Reverts table name back to system_setting in all database drivers (MySQL, PostgreSQL, SQLite)
- Removes migration files for the system_setting rename
- Reverts LATEST.sql files to use system_setting table
2026-01-07 20:38:02 +08:00
Johnny
e75862de31 chore: fix tests 2026-01-07 09:17:34 +08:00
Johnny
64b487d4af chore: fix seed data 2026-01-07 08:34:42 +08:00
Johnny
d326c71078 refactor(db): rename tables for clarity - resource→attachment, system_setting→instance_setting 2026-01-06 23:36:42 +08:00
Johnny
d68ca84832 fix: delete unused attachments by using filter 2026-01-05 22:43:57 +08:00
Johnny
bd02de9895
chore: add store tests (#5397) 2025-12-31 21:26:35 +08:00
Johnny
5d677828a6 refactor: remove NSFW content settings from instance configuration 2025-12-29 20:13:54 +08:00
Johnny
78aa41336a feat: implement attachment filtering functionality 2025-12-28 18:47:59 +08:00
Steven
d0c3908168 refactor: remove deprecated Sessions and AccessTokens settings
- Remove ListSessions and RevokeSession RPC endpoints
- Remove Session message and SessionsSetting from UserSetting
- Remove ACCESS_TOKENS key and AccessTokensSetting
- Update references to use RefreshTokensUserSetting with its own ClientInfo
- Remove UserSessionsSection frontend component
- Clean up user store to remove session and access token settings
- Regenerate protobuf files

The system now uses:
- REFRESH_TOKENS for session management with sliding expiration
- PERSONAL_ACCESS_TOKENS for long-lived API tokens
2025-12-19 08:35:57 +08:00
Johnny
7932f6d0d0
refactor: user auth improvements (#5360) 2025-12-18 18:15:51 +08:00
Johnny
310590b278 chore: fix golang checks 2025-12-16 22:53:36 +08:00
Johnny
40e129b8af refactor(auth): streamline session authentication and cookie handling 2025-12-16 22:23:59 +08:00
Steven
0610257562 refactor(store): remove deprecated migration_history table and backward compatibility code
Complete removal of migration_history system in favor of instance_setting based schema versioning.

Changes:
- Remove migration_history table creation from all LATEST.sql files
- Delete all migration_history model and implementation files (~300 lines)
- Remove FindMigrationHistoryList and UpsertMigrationHistory from Driver interface
- Replace complex backward compatibility functions with simple version check
- Update health check to use instance_setting instead of migration_history
- Simplify checkMinimumUpgradeVersion to detect pre-v0.22 installations

Breaking change:
Users on versions < v0.22.0 (May 2024) must upgrade to v0.25.x first before upgrading to this version.
Clear error message with upgrade instructions will be shown for old installations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 22:54:30 +08:00
Steven
d1492007ab fix(store): filter inbox notifications by message type at database level
Add MessageType filter to FindInbox to exclude legacy VERSION_UPDATE
notifications from inbox queries. This resolves the issue where users
saw notification counts but no items displayed, as VERSION_UPDATE
entries cannot be rendered in the new UserNotification API.

Changes:
- Add MessageType field to FindInbox struct for database-level filtering
- Implement JSON extraction filters in SQLite, MySQL, and PostgreSQL drivers
- Update ListUserNotifications to filter MEMO_COMMENT type at store level

This approach improves performance by filtering at the database rather
than in application code, reducing unnecessary data transfer for users
with many legacy inbox entries.

Fixes #5278

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 08:43:05 +08:00
Steven
07a030ddfd fix(postgres): update tag filtering SQL to ensure proper type casting for LIKE comparisons 2025-11-26 23:04:07 +08:00
Steven
50f49fc00d chore: update demo data 2025-11-26 21:58:56 +08:00
Steven
424f11f227 fix(store): fix PostgreSQL tag filtering type inference error
Resolves issue where tag filtering in PostgreSQL databases failed with "operator does not exist: jsonb ~~ unknown" error. The hierarchical tag filtering feature introduced in commit 5e47f25b generated SQL with implicit type placeholders that PostgreSQL couldn't infer.

The fix explicitly casts the LIKE comparison placeholder to text (::text) in the PostgreSQL dialect, ensuring proper type resolution for the query parameter.

Fixes #5275

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-26 07:40:39 +08:00
Steven
d1b2c0308b chore: add demo user access token for testing purposes 2025-11-11 08:31:05 +08:00
boojack
21d31e3609
fix(security): implement security review recommendations (#5228)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-06 23:32:27 +08:00
Steven
4c1d1c70d1 refactor: rename workspace to instance throughout codebase
Remove work-related terminology by renaming "workspace" to "instance"
across the entire application. This change better reflects that Memos
is a self-hosted tool suitable for personal and non-work use cases.

Breaking Changes:
- API endpoints: /api/v1/workspace/* → /api/v1/instance/*
- gRPC service: WorkspaceService → InstanceService
- Proto types: WorkspaceSetting → InstanceSetting
- Frontend translation keys: workspace-section → instance-section

Backend Changes:
- Renamed proto definitions and regenerated code
- Updated all store layer methods and database drivers
- Renamed service implementations and API handlers
- Updated cache from workspaceSettingCache to instanceSettingCache

Frontend Changes:
- Renamed service client: workspaceServiceClient → instanceServiceClient
- Updated all React components and state management
- Refactored stores: workspace.ts → instance.ts
- Updated all 32 locale translation files

All tests pass and both backend and frontend build successfully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 23:35:35 +08:00
Steven
bc1550e926 refactor(api): migrate inbox functionality to user notifications
- Remove standalone InboxService and move functionality to UserService
- Rename inbox to user notifications for better API consistency
- Add ListUserNotifications, UpdateUserNotification, DeleteUserNotification methods
- Update frontend components to use new notification endpoints
- Update store layer to support new notification model

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 08:33:09 +08:00
Steven
5e47f25bf5 feat(store): add hierarchical tag filtering support
Tag filters now support hierarchical matching where searching for a tag (e.g., "book") will match both the exact tag and any tags with that prefix (e.g., "book/fiction", "book/non-fiction"). This applies across all database backends (SQLite, MySQL, PostgreSQL) with corresponding test updates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 00:21:53 +08:00
Steven
f635d85bcf chore: fix reactions seed data
- Add comprehensive inline documentation for auth services
- Document session-based and token-based authentication flows
- Clarify authentication priority and validation logic
- Add detailed comments for JWT token structure and claims
- Fix reactions seed data to use memo UIDs instead of numeric IDs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-27 20:27:27 +08:00
Claude
f241b590a2 chore: update demo data 2025-10-27 09:20:26 +08:00
Steven
e0b1153269 fix(web): resolve MobX observable reactivity issue in filter computation
Fixes filtering functionality that was broken due to improper use of
useMemo with MobX observables. The issue occurred because useMemo's
dependency array uses reference equality, but MobX observable arrays
are mutated in place (reference doesn't change when items are added/removed).

Changes:
- Remove useMemo from filter computation in Home, UserProfile, and Archived pages
- Calculate filters directly in render since components are already MobX observers
- Fix typo: memoFitler -> memoFilter in Archived.tsx

This ensures filters are recalculated whenever memoFilterStore.filters changes,
making tag clicks and other filter interactions work correctly.

Fixes #5189

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 06:59:13 +08:00
Steven
46ce0bc62e fix(store): correct PostgreSQL placeholder generation in IN clauses
Fixes a regression introduced in v0.25.2 where PostgreSQL IN clause
placeholders were not properly incremented, causing all parameters to
use the same placeholder index (e.g., $1, $1, $1 instead of $1, $2, $3).

This bug affected:
- ListReactions (ContentIDList) - caused "failed to list reactions" errors
- ListAttachments (MemoIDList)
- ListMemos (IDList and UIDList)

The fix combines placeholder generation and argument appending into a
single loop to ensure proper incrementing.

Fixes #5188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 06:47:06 +08:00