Compare commits

...

4 Commits

Author SHA1 Message Date
140e22096f fix: correct text search CTE parameter numbering 2025-12-05 16:29:12 +05:30
8d8c22d781 Add support for extra ORDER BY fields in RowNumber methods
- Updated RowNumber, RowNumberDesc, RowNumberPartionBy, and RowNumberDescPartionBy to accept variadic extraOrderBy parameters
- Fixed bug in RowNumberDesc that was incorrectly using ASC instead of DESC
- Enhanced rowNumber internal function to build ORDER BY clause with primary field and additional fields
- Backwards compatible - existing code continues to work without extra fields
- Added CLAUDE.md documentation for future Claude Code instances
2025-11-29 13:53:11 +05:30
c2cf7ff088 removed file timestamping 2025-11-23 13:24:09 +05:30
cc7e6b7b3f Update dependencies 2025-11-22 16:47:32 +05:30
6 changed files with 363 additions and 23 deletions

314
CLAUDE.md Normal file
View File

@@ -0,0 +1,314 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**pgm** is a lightweight, type-safe PostgreSQL query builder for Go, built on top of jackc/pgx. It generates Go code from SQL schema files, enabling compile-time safety for database queries without the overhead of traditional ORMs.
**Core Philosophy:**
- Schema defined in SQL (not Go)
- Minimal code generation (only table/column definitions)
- Users provide their own models (no forced abstractions)
- Type-safe query building with fluent API
- Zero reflection for maximum performance
## Development Commands
### Building
```bash
# Build CLI with version from git tags
make build
# Build with specific version
make build VERSION=v1.2.3
# Install to GOPATH/bin
make install
# Check current version
make version
pgm -version
```
### Testing
```bash
# Run all tests in playground
make test
# Run benchmarks for SELECT queries
make bench-select
# Run code generator on playground schema
make run
```
### Code Generation
```bash
# Generate Go code from SQL schema
pgm -o ./db ./schema.sql
# The generator expects a SQL schema file and outputs:
# - One package per table in the output directory
# - Each package contains table and column definitions
# Example output: db/users/users.go, db/posts/posts.go
```
## Architecture
### Core Components
#### 1. Query Builder System (qry_*.go files)
The query builder uses a **mutable, stateful design** for conditional query building:
- **qry_select.go**: SELECT queries with joins, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, OFFSET
- **qry_insert.go**: INSERT queries with RETURNING and UPSERT (ON CONFLICT) support
- **qry_update.go**: UPDATE queries with conditional WHERE clauses
- **qry_delete.go**: DELETE queries with WHERE conditions
**Important:** Query builders accumulate state with each method call. They are designed for conditional building within a single query but should NOT be reused across multiple separate queries.
```go
// ✅ CORRECT - Conditional building
query := users.User.Select(users.ID, users.Email)
if nameFilter != "" {
query = query.Where(users.Name.Like("%" + nameFilter + "%"))
}
err := query.First(ctx, &id, &email)
// ❌ WRONG - Reusing builder across separate queries
baseQuery := users.User.Select(users.ID)
baseQuery.Where(users.ID.Eq(1)).First(ctx, &id1) // Adds ID=1
baseQuery.Where(users.Status.Eq(2)).First(ctx, &id2) // Has BOTH conditions!
```
Query builders are NOT thread-safe. Each goroutine must create its own query instance.
#### 2. Field System (pgm_field.go)
Fields are type-safe column references with:
- Comparison operators: `Eq()`, `NotEq()`, `Gt()`, `Lt()`, `Gte()`, `Lte()`
- Pattern matching: `Like()`, `ILike()`, `LikeFold()`, `EqFold()`
- NULL checks: `IsNull()`, `IsNotNull()`
- Array operations: `Any()`, `NotAny()`
- Aggregate functions: `Count()`, `Sum()`, `Avg()`, `Min()`, `Max()`
- String functions: `Lower()`, `Upper()`, `Trim()`, `StringEscape()`
- Special functions: `ConcatWs()`, `StringAgg()`, `DateTrunc()`, `RowNumber()`
**Security:** Functions like `ConcatWs()`, `StringAgg()`, and `DateTrunc()` validate inputs and escape SQL strings. They use allowlists for parameters like date truncation levels.
#### 3. Connection Pool (pgm.go)
Global connection pool using pgxpool with atomic pointer for thread safety:
- `InitPool(Config)`: Initialize once at startup (panics if called multiple times or with invalid config)
- `GetPool()`: Retrieve pool instance (panics if not initialized)
- `ClosePool()`: Graceful shutdown
- `BeginTx(ctx)`: Start transactions
The pool is stored in an `atomic.Pointer[pgxpool.Pool]` for lock-free concurrent access.
#### 4. Code Generator (cmd/)
Parses SQL schema files and generates Go code:
- **cmd/main.go**: CLI entry point with version flag support
- **cmd/parse.go**: Regex-based SQL parser (has known limitations)
- **cmd/generate.go**: Code generation with go/format for proper formatting
- **cmd/version.go**: Version string generation from build-time ldflags
**Parser Limitations:**
- No multi-line comments (`/* */`)
- Limited support for complex data types (arrays, JSON, JSONB)
- No advanced PostgreSQL features (PARTITION BY, INHERITS)
- Some constraints (CHECK, EXCLUDE) not parsed
Generated files include a header comment with version and timestamp:
```go
// Code generated by code.patial.tech/go/pgm/cmd v1.2.3 on 2025-11-16 04:05:43 DO NOT EDIT.
```
#### 5. Table System (pgm_table.go)
Minimal table metadata:
```go
type Table struct {
Name string // Table name
FieldCount int // Number of columns
PK []string // Primary key columns
DerivedTable Query // For subqueries/CTEs
}
```
Provides factory methods: `Select()`, `Insert()`, `Update()`, `Delete()`
### String Builder Pool
Uses `sync.Pool` for efficient string building (qry.go):
```go
var sbPool = sync.Pool{
New: func() any { return new(strings.Builder) }
}
```
All query builders use `getSB()` / `putSB()` to reduce allocations.
### Error Handling
Custom errors in pgm.go:
- `ErrConnStringMissing`: Connection string validation
- `ErrInitTX`: Transaction initialization failure
- `ErrCommitTX`: Transaction commit failure
- `ErrNoRows`: Wrapper for pgx.ErrNoRows
Use `pgm.IsNotFound(err)` to check for no rows errors.
### Generated Code Structure
For a schema with `users` and `posts` tables:
```
db/
├── schema.go # Table definitions and DerivedTable helper
├── user/
│ └── users.go # User table columns as constants
├── post/
│ └── posts.go # Post table columns as constants
└── ...
```
Each table file exports constants like:
```go
const (
All pgm.Field = "users.*"
ID pgm.Field = "users.id"
Email pgm.Field = "users.email"
// ... all columns
)
```
### Naming Conventions
- **Table pluralization**: `pluralToSingular()` in cmd/generate.go handles irregular plurals (people→person, children→child) and common patterns (-ies→-y, -ves→-fe, -es→-e, -s→"")
- **Field naming**: Snake_case columns converted to PascalCase (first_name → FirstName)
- **ID suffix**: Fields ending in `_id` become `ID` not `Id` (user_id → UserID)
## Key Implementation Details
### Query Building Strategy
1. **Pre-allocation**: Queries estimate final string length via `averageLen()` methods to reduce allocations
2. **Conditional accumulation**: All clauses stored in slices, built on demand
3. **Parameterized queries**: Uses PostgreSQL numbered parameters (`$1`, `$2`, etc.)
4. **Builder methods return interfaces**: Enforces correct method call sequences at compile time
Example type progression:
```go
SelectClause WhereClause AfterWhere OrderByClause Query First/All
```
### Transaction Handling
Pattern used throughout:
```go
tx, err := pgm.BeginTx(ctx)
if err != nil { return err }
defer tx.Rollback(ctx) // Safe to call after commit
// ... operations using ExecTx, FirstTx, AllTx methods ...
return tx.Commit(ctx)
```
All query execution methods have `Tx` variants that accept `pgx.Tx`.
### Full-Text Search Helpers
Helper functions for PostgreSQL tsvector queries:
- `TsAndQuery()`: AND operator between terms
- `TsPrefixAndQuery()`: AND with prefix matching `:*`
- `TsOrQuery()`: OR operator
- `TsPrefixOrQuery()`: OR with prefix matching
Used with tsvector columns via `Field.TsQuery()`.
## Testing
Tests are in the `playground/` directory:
- **playground/schema.sql**: Test database schema
- **playground/db/**: Generated code from schema
- **playground/*_test.go**: Integration tests for SELECT, INSERT, UPDATE, DELETE
- **playground/local_select_test.go**: Additional SELECT test cases
Tests require a running PostgreSQL instance. The playground uses the generated code to verify the query builder works correctly.
## Version Management
Version is injected at build time via ldflags:
```bash
go build -ldflags "-X main.version=v1.2.3" ./cmd
```
The Makefile automatically extracts version from git tags:
```makefile
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo "dev")
```
This version appears in:
- CLI `pgm -version` output
- Generated file headers
- Version string formatting in cmd/version.go
## Common Patterns
### Connection Pool Initialization
Always initialize once at startup and defer cleanup:
```go
func main() {
pgm.InitPool(pgm.Config{
ConnString: os.Getenv("DATABASE_URL"),
MaxConns: 25,
MinConns: 5,
})
defer pgm.ClosePool()
// ...
}
```
### Safe Query Execution
Check for no rows using the helper:
```go
err := users.User.Select(users.Email).Where(users.ID.Eq(id)).First(ctx, &email)
if pgm.IsNotFound(err) {
// Handle not found case
}
if err != nil {
// Handle other errors
}
```
### Validation Requirements
- UPDATE requires at least one `Set()` call
- INSERT requires at least one `Set()` call
- DELETE without `Where()` deletes ALL rows (dangerous!)
All execution methods validate these requirements and return descriptive errors.
## Performance Considerations
- **sync.Pool**: Reuses string builders across queries
- **Pre-allocation**: Queries pre-calculate buffer sizes
- **Zero reflection**: Direct field access, no runtime type inspection
- **pgxpool**: Leverages jackc/pgx's efficient connection pooling
- **Direct scanning**: Users scan into their own types, no intermediate mapping
## Security Notes
- All queries use parameterized statements (PostgreSQL numbered parameters)
- Field validation in functions like `DateTrunc()` uses allowlists
- SQL string escaping in `escapeSQLString()` for literal values
- Identifier validation via regex in `validateSQLIdentifier()`
- Connection pool configuration validates against negative values and invalid ranges

View File

@@ -9,7 +9,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"time"
"golang.org/x/text/cases" "golang.org/x/text/cases"
"golang.org/x/text/language" "golang.org/x/text/language"
@@ -41,8 +40,9 @@ func generate(scheamPath, outDir string) error {
// schema.go will hold all tables info // schema.go will hold all tables info
var sb strings.Builder var sb strings.Builder
sb.WriteString(fmt.Sprintf("// Code generated by code.patial.tech/go/pgm/cmd %s on %s DO NOT EDIT.\n\n", sb.WriteString(
GetVersionString(), time.Now().Format("2006-01-02 15:04:05"))) fmt.Sprintf("// Code generated by code.patial.tech/go/pgm/cmd %s DO NOT EDIT.\n\n", GetVersionString()),
)
sb.WriteString(fmt.Sprintf("package %s \n", filepath.Base(outDir))) sb.WriteString(fmt.Sprintf("package %s \n", filepath.Base(outDir)))
sb.WriteString(` sb.WriteString(`
import "code.patial.tech/go/pgm" import "code.patial.tech/go/pgm"
@@ -107,8 +107,9 @@ func generate(scheamPath, outDir string) error {
func writeColFile(tblName string, cols []*Column, outDir string, caser cases.Caser) error { func writeColFile(tblName string, cols []*Column, outDir string, caser cases.Caser) error {
var sb strings.Builder var sb strings.Builder
sb.WriteString(fmt.Sprintf("// Code generated by code.patial.tech/go/pgm/cmd %s on %s DO NOT EDIT.\n\n", sb.WriteString(
GetVersionString(), time.Now().Format("2006-01-02 15:04:05"))) fmt.Sprintf("// Code generated by code.patial.tech/go/pgm/cmd %s DO NOT EDIT.\n\n", GetVersionString()),
)
sb.WriteString(fmt.Sprintf("package %s\n\n", filepath.Base(outDir))) sb.WriteString(fmt.Sprintf("package %s\n\n", filepath.Base(outDir)))
sb.WriteString(fmt.Sprintf("import %q\n\n", "code.patial.tech/go/pgm")) sb.WriteString(fmt.Sprintf("import %q\n\n", "code.patial.tech/go/pgm"))
sb.WriteString("const (") sb.WriteString("const (")

8
go.mod
View File

@@ -3,14 +3,14 @@ module code.patial.tech/go/pgm
go 1.24.5 go 1.24.5
require ( require (
github.com/jackc/pgx/v5 v5.7.5 github.com/jackc/pgx/v5 v5.7.6
golang.org/x/text v0.27.0 golang.org/x/text v0.31.0
) )
require ( require (
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/crypto v0.40.0 // indirect golang.org/x/crypto v0.45.0 // indirect
golang.org/x/sync v0.16.0 // indirect golang.org/x/sync v0.18.0 // indirect
) )

8
go.sum
View File

@@ -7,6 +7,8 @@ github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7Ulw
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM= github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.5 h1:JHGfMnQY+IEtGM63d+NGMjoRpysB2JBwDr5fsngwmJs= github.com/jackc/pgx/v5 v5.7.5 h1:JHGfMnQY+IEtGM63d+NGMjoRpysB2JBwDr5fsngwmJs=
github.com/jackc/pgx/v5 v5.7.5/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M= github.com/jackc/pgx/v5 v5.7.5/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -18,10 +20,16 @@ github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKs
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM= golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY= golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4= golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU= golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

View File

@@ -152,24 +152,24 @@ func (f Field) Desc() Field {
return Field(f.String() + " DESC") return Field(f.String() + " DESC")
} }
func (f Field) RowNumber(as string) Field { func (f Field) RowNumber(as string, extraOrderBy ...Field) Field {
return rowNumber(&f, nil, true, as) return rowNumber(&f, nil, true, as, extraOrderBy...)
} }
func (f Field) RowNumberDesc(as string) Field { func (f Field) RowNumberDesc(as string, extraOrderBy ...Field) Field {
return rowNumber(&f, nil, true, as) return rowNumber(&f, nil, false, as, extraOrderBy...)
} }
// RowNumberPartionBy in ascending order // RowNumberPartionBy in ascending order
func (f Field) RowNumberPartionBy(partition Field, as string) Field { func (f Field) RowNumberPartionBy(partition Field, as string, extraOrderBy ...Field) Field {
return rowNumber(&f, &partition, true, as) return rowNumber(&f, &partition, true, as, extraOrderBy...)
} }
func (f Field) RowNumberDescPartionBy(partition Field, as string) Field { func (f Field) RowNumberDescPartionBy(partition Field, as string, extraOrderBy ...Field) Field {
return rowNumber(&f, &partition, false, as) return rowNumber(&f, &partition, false, as, extraOrderBy...)
} }
func rowNumber(f, partition *Field, isAsc bool, as string) Field { func rowNumber(f, partition *Field, isAsc bool, as string, extraOrderBy ...Field) Field {
// Validate as parameter is a valid SQL identifier // Validate as parameter is a valid SQL identifier
if as != "" { if as != "" {
if err := validateSQLIdentifier(as); err != nil { if err := validateSQLIdentifier(as); err != nil {
@@ -188,11 +188,27 @@ func rowNumber(f, partition *Field, isAsc bool, as string) Field {
} }
col := f.String() col := f.String()
if partition != nil {
return Field("ROW_NUMBER() OVER (PARTITION BY " + partition.String() + " ORDER BY " + col + orderBy + ") AS " + as) // Build ORDER BY clause with primary field and extra fields
sb := getSB()
defer putSB(sb)
sb.WriteString(col)
sb.WriteString(orderBy)
// Add extra ORDER BY fields
for _, extra := range extraOrderBy {
sb.WriteString(", ")
sb.WriteString(extra.String())
} }
return Field("ROW_NUMBER() OVER (ORDER BY " + col + orderBy + ") AS " + as) orderByClause := sb.String()
if partition != nil {
return Field("ROW_NUMBER() OVER (PARTITION BY " + partition.String() + " ORDER BY " + orderByClause + ") AS " + as)
}
return Field("ROW_NUMBER() OVER (ORDER BY " + orderByClause + ") AS " + as)
} }
func (f Field) IsNull() Conditioner { func (f Field) IsNull() Conditioner {

View File

@@ -335,8 +335,9 @@ func (q *selectQry) Build(needArgs bool) (qry string, args []any) {
if q.textSearch != nil { if q.textSearch != nil {
var ts = q.textSearch var ts = q.textSearch
q.args = slices.Insert(q.args, 0, any(ts.value)) q.args = append(q.args, any(ts.value))
sb.WriteString("WITH " + ts.name + " AS (SELECT to_tsquery('english', $1) AS " + ts.alias + ") ") sb.WriteString("WITH " + ts.name + " AS (SELECT to_tsquery('english', $" +
strconv.Itoa(len(q.args)) + ") AS " + ts.alias + ") ")
} }
// SELECT // SELECT