Compare commits

...

1 Commits
v0.4.2 ... main

Author SHA1 Message Date
8d8c22d781 Add support for extra ORDER BY fields in RowNumber methods
- Updated RowNumber, RowNumberDesc, RowNumberPartionBy, and RowNumberDescPartionBy to accept variadic extraOrderBy parameters
- Fixed bug in RowNumberDesc that was incorrectly using ASC instead of DESC
- Enhanced rowNumber internal function to build ORDER BY clause with primary field and additional fields
- Backwards compatible - existing code continues to work without extra fields
- Added CLAUDE.md documentation for future Claude Code instances
2025-11-29 13:53:11 +05:30
2 changed files with 342 additions and 12 deletions

314
CLAUDE.md Normal file
View File

@@ -0,0 +1,314 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**pgm** is a lightweight, type-safe PostgreSQL query builder for Go, built on top of jackc/pgx. It generates Go code from SQL schema files, enabling compile-time safety for database queries without the overhead of traditional ORMs.
**Core Philosophy:**
- Schema defined in SQL (not Go)
- Minimal code generation (only table/column definitions)
- Users provide their own models (no forced abstractions)
- Type-safe query building with fluent API
- Zero reflection for maximum performance
## Development Commands
### Building
```bash
# Build CLI with version from git tags
make build
# Build with specific version
make build VERSION=v1.2.3
# Install to GOPATH/bin
make install
# Check current version
make version
pgm -version
```
### Testing
```bash
# Run all tests in playground
make test
# Run benchmarks for SELECT queries
make bench-select
# Run code generator on playground schema
make run
```
### Code Generation
```bash
# Generate Go code from SQL schema
pgm -o ./db ./schema.sql
# The generator expects a SQL schema file and outputs:
# - One package per table in the output directory
# - Each package contains table and column definitions
# Example output: db/users/users.go, db/posts/posts.go
```
## Architecture
### Core Components
#### 1. Query Builder System (qry_*.go files)
The query builder uses a **mutable, stateful design** for conditional query building:
- **qry_select.go**: SELECT queries with joins, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, OFFSET
- **qry_insert.go**: INSERT queries with RETURNING and UPSERT (ON CONFLICT) support
- **qry_update.go**: UPDATE queries with conditional WHERE clauses
- **qry_delete.go**: DELETE queries with WHERE conditions
**Important:** Query builders accumulate state with each method call. They are designed for conditional building within a single query but should NOT be reused across multiple separate queries.
```go
// ✅ CORRECT - Conditional building
query := users.User.Select(users.ID, users.Email)
if nameFilter != "" {
query = query.Where(users.Name.Like("%" + nameFilter + "%"))
}
err := query.First(ctx, &id, &email)
// ❌ WRONG - Reusing builder across separate queries
baseQuery := users.User.Select(users.ID)
baseQuery.Where(users.ID.Eq(1)).First(ctx, &id1) // Adds ID=1
baseQuery.Where(users.Status.Eq(2)).First(ctx, &id2) // Has BOTH conditions!
```
Query builders are NOT thread-safe. Each goroutine must create its own query instance.
#### 2. Field System (pgm_field.go)
Fields are type-safe column references with:
- Comparison operators: `Eq()`, `NotEq()`, `Gt()`, `Lt()`, `Gte()`, `Lte()`
- Pattern matching: `Like()`, `ILike()`, `LikeFold()`, `EqFold()`
- NULL checks: `IsNull()`, `IsNotNull()`
- Array operations: `Any()`, `NotAny()`
- Aggregate functions: `Count()`, `Sum()`, `Avg()`, `Min()`, `Max()`
- String functions: `Lower()`, `Upper()`, `Trim()`, `StringEscape()`
- Special functions: `ConcatWs()`, `StringAgg()`, `DateTrunc()`, `RowNumber()`
**Security:** Functions like `ConcatWs()`, `StringAgg()`, and `DateTrunc()` validate inputs and escape SQL strings. They use allowlists for parameters like date truncation levels.
#### 3. Connection Pool (pgm.go)
Global connection pool using pgxpool with atomic pointer for thread safety:
- `InitPool(Config)`: Initialize once at startup (panics if called multiple times or with invalid config)
- `GetPool()`: Retrieve pool instance (panics if not initialized)
- `ClosePool()`: Graceful shutdown
- `BeginTx(ctx)`: Start transactions
The pool is stored in an `atomic.Pointer[pgxpool.Pool]` for lock-free concurrent access.
#### 4. Code Generator (cmd/)
Parses SQL schema files and generates Go code:
- **cmd/main.go**: CLI entry point with version flag support
- **cmd/parse.go**: Regex-based SQL parser (has known limitations)
- **cmd/generate.go**: Code generation with go/format for proper formatting
- **cmd/version.go**: Version string generation from build-time ldflags
**Parser Limitations:**
- No multi-line comments (`/* */`)
- Limited support for complex data types (arrays, JSON, JSONB)
- No advanced PostgreSQL features (PARTITION BY, INHERITS)
- Some constraints (CHECK, EXCLUDE) not parsed
Generated files include a header comment with version and timestamp:
```go
// Code generated by code.patial.tech/go/pgm/cmd v1.2.3 on 2025-11-16 04:05:43 DO NOT EDIT.
```
#### 5. Table System (pgm_table.go)
Minimal table metadata:
```go
type Table struct {
Name string // Table name
FieldCount int // Number of columns
PK []string // Primary key columns
DerivedTable Query // For subqueries/CTEs
}
```
Provides factory methods: `Select()`, `Insert()`, `Update()`, `Delete()`
### String Builder Pool
Uses `sync.Pool` for efficient string building (qry.go):
```go
var sbPool = sync.Pool{
New: func() any { return new(strings.Builder) }
}
```
All query builders use `getSB()` / `putSB()` to reduce allocations.
### Error Handling
Custom errors in pgm.go:
- `ErrConnStringMissing`: Connection string validation
- `ErrInitTX`: Transaction initialization failure
- `ErrCommitTX`: Transaction commit failure
- `ErrNoRows`: Wrapper for pgx.ErrNoRows
Use `pgm.IsNotFound(err)` to check for no rows errors.
### Generated Code Structure
For a schema with `users` and `posts` tables:
```
db/
├── schema.go # Table definitions and DerivedTable helper
├── user/
│ └── users.go # User table columns as constants
├── post/
│ └── posts.go # Post table columns as constants
└── ...
```
Each table file exports constants like:
```go
const (
All pgm.Field = "users.*"
ID pgm.Field = "users.id"
Email pgm.Field = "users.email"
// ... all columns
)
```
### Naming Conventions
- **Table pluralization**: `pluralToSingular()` in cmd/generate.go handles irregular plurals (people→person, children→child) and common patterns (-ies→-y, -ves→-fe, -es→-e, -s→"")
- **Field naming**: Snake_case columns converted to PascalCase (first_name → FirstName)
- **ID suffix**: Fields ending in `_id` become `ID` not `Id` (user_id → UserID)
## Key Implementation Details
### Query Building Strategy
1. **Pre-allocation**: Queries estimate final string length via `averageLen()` methods to reduce allocations
2. **Conditional accumulation**: All clauses stored in slices, built on demand
3. **Parameterized queries**: Uses PostgreSQL numbered parameters (`$1`, `$2`, etc.)
4. **Builder methods return interfaces**: Enforces correct method call sequences at compile time
Example type progression:
```go
SelectClause WhereClause AfterWhere OrderByClause Query First/All
```
### Transaction Handling
Pattern used throughout:
```go
tx, err := pgm.BeginTx(ctx)
if err != nil { return err }
defer tx.Rollback(ctx) // Safe to call after commit
// ... operations using ExecTx, FirstTx, AllTx methods ...
return tx.Commit(ctx)
```
All query execution methods have `Tx` variants that accept `pgx.Tx`.
### Full-Text Search Helpers
Helper functions for PostgreSQL tsvector queries:
- `TsAndQuery()`: AND operator between terms
- `TsPrefixAndQuery()`: AND with prefix matching `:*`
- `TsOrQuery()`: OR operator
- `TsPrefixOrQuery()`: OR with prefix matching
Used with tsvector columns via `Field.TsQuery()`.
## Testing
Tests are in the `playground/` directory:
- **playground/schema.sql**: Test database schema
- **playground/db/**: Generated code from schema
- **playground/*_test.go**: Integration tests for SELECT, INSERT, UPDATE, DELETE
- **playground/local_select_test.go**: Additional SELECT test cases
Tests require a running PostgreSQL instance. The playground uses the generated code to verify the query builder works correctly.
## Version Management
Version is injected at build time via ldflags:
```bash
go build -ldflags "-X main.version=v1.2.3" ./cmd
```
The Makefile automatically extracts version from git tags:
```makefile
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
```
This version appears in:
- CLI `pgm -version` output
- Generated file headers
- Version string formatting in cmd/version.go
## Common Patterns
### Connection Pool Initialization
Always initialize once at startup and defer cleanup:
```go
func main() {
pgm.InitPool(pgm.Config{
ConnString: os.Getenv("DATABASE_URL"),
MaxConns: 25,
MinConns: 5,
})
defer pgm.ClosePool()
// ...
}
```
### Safe Query Execution
Check for no rows using the helper:
```go
err := users.User.Select(users.Email).Where(users.ID.Eq(id)).First(ctx, &email)
if pgm.IsNotFound(err) {
// Handle not found case
}
if err != nil {
// Handle other errors
}
```
### Validation Requirements
- UPDATE requires at least one `Set()` call
- INSERT requires at least one `Set()` call
- DELETE without `Where()` deletes ALL rows (dangerous!)
All execution methods validate these requirements and return descriptive errors.
## Performance Considerations
- **sync.Pool**: Reuses string builders across queries
- **Pre-allocation**: Queries pre-calculate buffer sizes
- **Zero reflection**: Direct field access, no runtime type inspection
- **pgxpool**: Leverages jackc/pgx's efficient connection pooling
- **Direct scanning**: Users scan into their own types, no intermediate mapping
## Security Notes
- All queries use parameterized statements (PostgreSQL numbered parameters)
- Field validation in functions like `DateTrunc()` uses allowlists
- SQL string escaping in `escapeSQLString()` for literal values
- Identifier validation via regex in `validateSQLIdentifier()`
- Connection pool configuration validates against negative values and invalid ranges

View File

@@ -152,24 +152,24 @@ func (f Field) Desc() Field {
return Field(f.String() + " DESC") return Field(f.String() + " DESC")
} }
func (f Field) RowNumber(as string) Field { func (f Field) RowNumber(as string, extraOrderBy ...Field) Field {
return rowNumber(&f, nil, true, as) return rowNumber(&f, nil, true, as, extraOrderBy...)
} }
func (f Field) RowNumberDesc(as string) Field { func (f Field) RowNumberDesc(as string, extraOrderBy ...Field) Field {
return rowNumber(&f, nil, true, as) return rowNumber(&f, nil, false, as, extraOrderBy...)
} }
// RowNumberPartionBy in ascending order // RowNumberPartionBy in ascending order
func (f Field) RowNumberPartionBy(partition Field, as string) Field { func (f Field) RowNumberPartionBy(partition Field, as string, extraOrderBy ...Field) Field {
return rowNumber(&f, &partition, true, as) return rowNumber(&f, &partition, true, as, extraOrderBy...)
} }
func (f Field) RowNumberDescPartionBy(partition Field, as string) Field { func (f Field) RowNumberDescPartionBy(partition Field, as string, extraOrderBy ...Field) Field {
return rowNumber(&f, &partition, false, as) return rowNumber(&f, &partition, false, as, extraOrderBy...)
} }
func rowNumber(f, partition *Field, isAsc bool, as string) Field { func rowNumber(f, partition *Field, isAsc bool, as string, extraOrderBy ...Field) Field {
// Validate as parameter is a valid SQL identifier // Validate as parameter is a valid SQL identifier
if as != "" { if as != "" {
if err := validateSQLIdentifier(as); err != nil { if err := validateSQLIdentifier(as); err != nil {
@@ -188,11 +188,27 @@ func rowNumber(f, partition *Field, isAsc bool, as string) Field {
} }
col := f.String() col := f.String()
if partition != nil {
return Field("ROW_NUMBER() OVER (PARTITION BY " + partition.String() + " ORDER BY " + col + orderBy + ") AS " + as) // Build ORDER BY clause with primary field and extra fields
sb := getSB()
defer putSB(sb)
sb.WriteString(col)
sb.WriteString(orderBy)
// Add extra ORDER BY fields
for _, extra := range extraOrderBy {
sb.WriteString(", ")
sb.WriteString(extra.String())
} }
return Field("ROW_NUMBER() OVER (ORDER BY " + col + orderBy + ") AS " + as) orderByClause := sb.String()
if partition != nil {
return Field("ROW_NUMBER() OVER (PARTITION BY " + partition.String() + " ORDER BY " + orderByClause + ") AS " + as)
}
return Field("ROW_NUMBER() OVER (ORDER BY " + orderByClause + ") AS " + as)
} }
func (f Field) IsNull() Conditioner { func (f Field) IsNull() Conditioner {