Compare commits
12 Commits
Author | SHA1 | Date | |
---|---|---|---|
8750f3ad95 | |||
ad1faf2056 | |||
525c64e678 | |||
5f0fdadb8b | |||
68263895f7 | |||
ee6cb445ab | |||
d07c25fe01 | |||
096480a3eb | |||
6c14441591 | |||
d95eea6636 | |||
63b71692b5 | |||
36e4145365 |
3
Makefile
3
Makefile
@@ -2,3 +2,6 @@ run:
|
||||
go run ./cmd -o ./example/db ./example/schema.sql
|
||||
bench-select:
|
||||
go test ./example -bench BenchmarkSelect -memprofile memprofile.out -cpuprofile profile.out
|
||||
|
||||
test:
|
||||
go test ./playground
|
||||
|
71
README.md
71
README.md
@@ -1,51 +1,57 @@
|
||||
# pgm - PostgreSQL query mapper
|
||||
A simple ORM that will work on top of [jackc/pgx](https://github.com/jackc/pgx) db connection pool.
|
||||
# pgm - PostgreSQL Query Mapper
|
||||
|
||||
A lightweight ORM built on top of [jackc/pgx](https://github.com/jackc/pgx) database connection pool.
|
||||
|
||||
## ORMs I Like in the Go Ecosystem
|
||||
|
||||
## ORMs in go eco system tha I like
|
||||
- [ent](https://github.com/ent/ent)
|
||||
- [sqlc](https://github.com/sqlc-dev/sqlc)
|
||||
|
||||
## Why not to use ent?
|
||||
ent is feature loadedm, you can easily define complex schemas, it has nice integration with graphQL.
|
||||
Very nice auto migration feature. Basically all the things you need in an ORM.
|
||||
## Why Not Use `ent`?
|
||||
|
||||
But to me its overkill for simple apps, I have seen ent related code is taking significatn space in app binary(feels bloated).
|
||||
`ent` is a feature-rich ORM with schema definition, automatic migrations, integration with `gqlgen` (GraphQL server), and more. It provides nearly everything you could want in an ORM.
|
||||
|
||||
However, it can be overkill. The generated code supports a wide range of features, many of which you may not use, significantly increasing the compiled binary size.
|
||||
|
||||
## Why not to use sqlc?
|
||||
sqlc is nice as well, but using it will start feeling that now your DB layer has its own Models and you need map you apps model with it or have no choice but to sart using DB layer models, which to me is not a good thing.
|
||||
## Why Not Use `sqlc`?
|
||||
|
||||
## Things that I am not happy with
|
||||
`sqlc` is a great tool, but it often feels like the database layer introduces its own models. This forces you to either map your application’s models to these database models or use the database models directly, which may not align with your application’s design.
|
||||
|
||||
- Auto migrations, at many points you will see ORM is either not providing or have a complex way to do the simple db schema related changes. Its like obscure db schema. you can simply tell and test it in sql query editor.
|
||||
## Issues with Existing ORMs
|
||||
|
||||
DB must be a in form sql statement that you see, fine tune and event run/test in sql query editor.
|
||||
Thers is lots of mature tools available in Go ecosystem, [dbmate](https://github.com/amacneil/dbmate) is one of them, its a nice tool to manage migrations, can be use in code or in cli
|
||||
Here are some common pain points with ORMs:
|
||||
|
||||
- To much extra code is generated for various condition and scenarios that you may not use
|
||||
- **Auto Migrations**: Many ORMs either lack robust migration support or implement complex methods for simple schema changes. This can obscure the database schema, making it harder to understand and maintain. A database schema should be defined in clear SQL statements that can be tested in a SQL query editor. Tools like [dbmate](https://github.com/amacneil/dbmate) provide a mature solution for managing migrations, usable via CLI or in code.
|
||||
|
||||
- Auto genearted models for select queries that will now force you to either use them or to map them to required model.
|
||||
- **Excessive Code Generation**: ORMs often generate excessive code for various conditions and scenarios, much of which goes unused.
|
||||
|
||||
## Okay, what to do now, plain old sql queries?
|
||||
Yes and No may hybrid. Plain old sql queries are not bad, just have have few concerns
|
||||
- **Generated Models for Queries**: Auto-generated models for `SELECT` queries force you to either adopt them or map them to your application’s models, adding complexity.
|
||||
|
||||
- Change in schema are not detected well.
|
||||
- Sql in jection issue if not using parameterized queries.
|
||||
## A Hybrid Approach: Plain SQL Queries with `pgm`
|
||||
|
||||
We can address these issues using pgm, pgm cli will help to cream light weight DB schema go files that will help in writing sql queries this can help keeping eye on schema changes, goal is not to hard code table and table.column names
|
||||
Plain SQL queries are not inherently bad but come with challenges:
|
||||
|
||||
- **Schema Change Detection**: Changes in the database schema are not easily detected.
|
||||
- **SQL Injection Risks**: Without parameterized queries, SQL injection becomes a concern.
|
||||
|
||||
`pgm` addresses these issues by providing a lightweight CLI tool that generates Go files for your database schema. These files help you write SQL queries while keeping track of schema changes, avoiding hardcoded table and column names.
|
||||
|
||||
## Generating `pgm` Schema Files
|
||||
|
||||
Run the following command to generate schema files:
|
||||
|
||||
## Generate pgm schema files
|
||||
run following command to generate pgm schema files
|
||||
```bash
|
||||
go run code.patial.tech/go/pgm/cmd -o ./db ./schema.sql
|
||||
go run code.partial.tech/go/pgm/cmd -o ./db ./schema.sql
|
||||
```
|
||||
It will create bunch of files and filders under `./db` directory
|
||||
|
||||
once you have the schama files created you can use `pgm` as
|
||||
|
||||
```go
|
||||
package main
|
||||
import "code.patial.tech/go/pgm"
|
||||
|
||||
import (
|
||||
"code.partial.tech/go/pgm"
|
||||
"myapp/db/user" // scham create by pgm/cmd
|
||||
)
|
||||
|
||||
type MyModel struct {
|
||||
ID string
|
||||
@@ -53,22 +59,23 @@ type MyModel struct {
|
||||
}
|
||||
|
||||
func main() {
|
||||
println("init pgx connection pool")
|
||||
println("Initializing pgx connection pool")
|
||||
pgm.InitPool(pgm.Config{
|
||||
ConnString: url,
|
||||
})
|
||||
|
||||
// Select query with first record to scan
|
||||
// it is assuming that schema is already created and is in "db" package and has User table init
|
||||
// Select query to fetch the first record
|
||||
// Assumes the schema is defined in the "db" package with a User table
|
||||
var v MyModel
|
||||
err := db.User.Select(user.ID, user.Email).
|
||||
Where(user.Email.Like("anki%")).
|
||||
First(context.TODO(), &v.Email, &v.ID)
|
||||
First(context.TODO(), &v.ID, &v.Email)
|
||||
if err != nil {
|
||||
println("error, ", err.Error())
|
||||
println("Error:", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
println("user email", v.Email)
|
||||
println("User email:", v.Email)
|
||||
}
|
||||
```
|
||||
|
||||
|
79
pgm.go
79
pgm.go
@@ -44,23 +44,73 @@ func (f Field) Count() Field {
|
||||
return Field("COUNT(" + f.String() + ")")
|
||||
}
|
||||
|
||||
// StringEscape will return a empty string for null value
|
||||
func (f Field) StringEscape() Field {
|
||||
return Field("COALESCE(" + f.String() + ", '')")
|
||||
}
|
||||
|
||||
// NumberEscape will return a zero string for null value
|
||||
func (f Field) NumberEscape() Field {
|
||||
return Field("COALESCE(" + f.String() + ", 0)")
|
||||
}
|
||||
|
||||
// BooleanEscape will return a false for null value
|
||||
func (f Field) BooleanEscape() Field {
|
||||
return Field("COALESCE(" + f.String() + ", FALSE)")
|
||||
}
|
||||
|
||||
// Avg fn wrapping of field
|
||||
func (f Field) Avg() Field {
|
||||
return Field("AVG(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Sum() Field {
|
||||
return Field("SUM(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Max() Field {
|
||||
return Field("MAX(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Min() Field {
|
||||
return Field("Min(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Lower() Field {
|
||||
return Field("LOWER(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Upper() Field {
|
||||
return Field("UPPER(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) Trim() Field {
|
||||
return Field("TRIM(" + f.String() + ")")
|
||||
}
|
||||
|
||||
func (f Field) IsNull() Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, op: " IS NULL", len: len(col) + 8}
|
||||
}
|
||||
|
||||
func (f Field) IsNotNull() Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, op: " IS NOT NULL", len: len(col) + 12}
|
||||
}
|
||||
|
||||
// EqualFold will use LOWER(column_name) = LOWER(val) for comparision
|
||||
func (f Field) EqFold(val string) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: "LOWER(" + col + ")", Val: val, op: " = LOWER($", action: CondActionNeedToClose, len: len(col) + 5}
|
||||
}
|
||||
|
||||
// Eq is equal
|
||||
func (f Field) Eq(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " = $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
// EqualFold will user LOWER() for comparision
|
||||
func (f Field) EqFold(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: "LOWER(" + col + ")", Val: val, op: " = LOWER($", action: CondActionNeedToClose, len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) NEq(val any) Conditioner {
|
||||
func (f Field) NotEq(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " != $", len: len(col) + 5}
|
||||
}
|
||||
@@ -70,11 +120,21 @@ func (f Field) Gt(val any) Conditioner {
|
||||
return &Cond{Field: col, Val: val, op: " > $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) Lt(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " < $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) Gte(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " >= $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) Lte(val any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " <= $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) Like(val string) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " LIKE $", len: len(f.String()) + 5}
|
||||
@@ -91,6 +151,11 @@ func (f Field) ILike(val string) Conditioner {
|
||||
return &Cond{Field: col, Val: val, op: " ILIKE $", len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) In(val ...any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " IN($", action: CondActionNeedToClose, len: len(col) + 5}
|
||||
}
|
||||
|
||||
func (f Field) NotIn(val ...any) Conditioner {
|
||||
col := f.String()
|
||||
return &Cond{Field: col, Val: val, op: " NOT IN($", action: CondActionNeedToClose, len: len(col) + 5}
|
||||
|
@@ -28,8 +28,9 @@ func TestQryBuilder2(t *testing.T) {
|
||||
),
|
||||
).
|
||||
Where(
|
||||
user.LastName.NEq(7),
|
||||
user.LastName.NotEq(7),
|
||||
user.Phone.Like("%123%"),
|
||||
user.UpdatedAt.IsNotNull(),
|
||||
user.Email.NotInSubQuery(db.User.Select(user.ID).Where(user.ID.Eq(123))),
|
||||
).
|
||||
Limit(10).
|
||||
@@ -39,7 +40,7 @@ func TestQryBuilder2(t *testing.T) {
|
||||
expected := "SELECT users.email, users.first_name FROM users JOIN user_sessions ON users.id = user_sessions.user_id" +
|
||||
" JOIN branch_users ON users.id = branch_users.user_id WHERE users.id = $1 AND (users.status_id = $2 OR users.updated_at = $3)" +
|
||||
" AND users.mfa_kind = $4 AND (users.first_name = $5 OR users.middle_name = $6) AND users.last_name != $7 AND users.phone" +
|
||||
" LIKE $8 AND users.email NOT IN(SELECT users.id FROM users WHERE users.id = $9) LIMIT 10 OFFSET 100"
|
||||
" LIKE $8 AND users.updated_at IS NOT NULL AND users.email NOT IN(SELECT users.id FROM users WHERE users.id = $9) LIMIT 10 OFFSET 100"
|
||||
if expected != got {
|
||||
t.Errorf("\nexpected: %q\ngot: %q", expected, got)
|
||||
}
|
||||
@@ -59,7 +60,31 @@ func TestSelectWithHaving(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// BenchmarkSelect-12 668817 1753 ns/op 4442 B/op 59 allocs/op
|
||||
func TestSelectWithJoin(t *testing.T) {
|
||||
got := db.User.Select(user.Email, user.FirstName).
|
||||
Join(db.UserSession, user.ID, usersession.UserID).
|
||||
LeftJoin(db.BranchUser, user.ID, branchuser.UserID, pgm.Or(branchuser.RoleID.Eq("1"), branchuser.RoleID.Eq("2"))).
|
||||
Where(
|
||||
user.ID.Eq(3),
|
||||
pgm.Or(
|
||||
user.StatusID.Eq(4),
|
||||
user.UpdatedAt.Eq(5),
|
||||
),
|
||||
).
|
||||
Limit(10).
|
||||
Offset(100).
|
||||
String()
|
||||
|
||||
expected := "SELECT users.email, users.first_name " +
|
||||
"FROM users JOIN user_sessions ON users.id = user_sessions.user_id " +
|
||||
"LEFT JOIN branch_users ON users.id = branch_users.user_id AND (branch_users.role_id = $1 OR branch_users.role_id = $2) " +
|
||||
"WHERE users.id = $3 AND (users.status_id = $4 OR users.updated_at = $5) " +
|
||||
"LIMIT 10 OFFSET 100"
|
||||
if expected != got {
|
||||
t.Errorf("\nexpected: %q\ngot: %q", expected, got)
|
||||
}
|
||||
}
|
||||
|
||||
// BenchmarkSelect-12 638901 1860 ns/op 4266 B/op 61 allocs/op
|
||||
func BenchmarkSelect(b *testing.B) {
|
||||
for b.Loop() {
|
||||
@@ -79,7 +104,7 @@ func BenchmarkSelect(b *testing.B) {
|
||||
),
|
||||
).
|
||||
Where(
|
||||
user.LastName.NEq(7),
|
||||
user.LastName.NotEq(7),
|
||||
user.Phone.Like("%123%"),
|
||||
user.Email.NotInSubQuery(db.User.Select(user.ID).Where(user.ID.Eq(123))),
|
||||
).
|
||||
|
@@ -17,7 +17,7 @@ func TestUpdateQuery(t *testing.T) {
|
||||
user.Email.Eq("aa@aa.com"),
|
||||
).
|
||||
Where(
|
||||
user.StatusID.NEq(1),
|
||||
user.StatusID.NotEq(1),
|
||||
).
|
||||
String()
|
||||
|
||||
@@ -38,7 +38,7 @@ func TestUpdateSetMap(t *testing.T) {
|
||||
user.Email.Eq("aa@aa.com"),
|
||||
).
|
||||
Where(
|
||||
user.StatusID.NEq(1),
|
||||
user.StatusID.NotEq(1),
|
||||
).
|
||||
String()
|
||||
|
||||
@@ -59,7 +59,7 @@ func BenchmarkUpdateQuery(b *testing.B) {
|
||||
user.Email.Eq("aa@aa.com"),
|
||||
).
|
||||
Where(
|
||||
user.StatusID.NEq(1),
|
||||
user.StatusID.NotEq(1),
|
||||
).
|
||||
String()
|
||||
}
|
||||
|
14
qry.go
14
qry.go
@@ -21,10 +21,10 @@ type (
|
||||
|
||||
SelectClause interface {
|
||||
// Join and Inner Join are same
|
||||
Join(m Table, t1Field, t2Field Field) SelectClause
|
||||
LeftJoin(m Table, t1Field, t2Field Field) SelectClause
|
||||
RightJoin(m Table, t1Field, t2Field Field) SelectClause
|
||||
FullJoin(m Table, t1Field, t2Field Field) SelectClause
|
||||
Join(m Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause
|
||||
LeftJoin(m Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause
|
||||
RightJoin(m Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause
|
||||
FullJoin(m Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause
|
||||
CrossJoin(m Table) SelectClause
|
||||
WhereClause
|
||||
OrderByClause
|
||||
@@ -208,13 +208,17 @@ func (cv *Cond) Condition(args *[]any, argIdx int) string {
|
||||
}
|
||||
|
||||
// 2. normal condition
|
||||
*args = append(*args, cv.Val)
|
||||
var op string
|
||||
if cv.Val != nil {
|
||||
*args = append(*args, cv.Val)
|
||||
if strings.HasSuffix(cv.op, "$") {
|
||||
op = cv.op + strconv.Itoa(argIdx+1)
|
||||
} else {
|
||||
op = strings.Replace(cv.op, "$", "$"+strconv.Itoa(argIdx+1), 1)
|
||||
}
|
||||
} else {
|
||||
op = cv.op
|
||||
}
|
||||
|
||||
if cv.action == CondActionNeedToClose {
|
||||
return cv.Field + op + ")"
|
||||
|
@@ -62,23 +62,46 @@ func (t Table) Select(field ...Field) SelectClause {
|
||||
return qb
|
||||
}
|
||||
|
||||
func (q *selectQry) Join(t Table, t1Field, t2Field Field) SelectClause {
|
||||
q.join = append(q.join, "JOIN "+t.Name+" ON "+t1Field.String()+" = "+t2Field.String())
|
||||
func (q *selectQry) Join(t Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause {
|
||||
return q.buildJoin(t, "JOIN", t1Field, t2Field, cond...)
|
||||
}
|
||||
|
||||
func (q *selectQry) LeftJoin(t Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause {
|
||||
return q.buildJoin(t, "LEFT JOIN", t1Field, t2Field, cond...)
|
||||
}
|
||||
|
||||
func (q *selectQry) RightJoin(t Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause {
|
||||
return q.buildJoin(t, "RIGHT JOIN", t1Field, t2Field, cond...)
|
||||
}
|
||||
|
||||
func (q *selectQry) FullJoin(t Table, t1Field, t2Field Field, cond ...Conditioner) SelectClause {
|
||||
return q.buildJoin(t, "FULL JOIN", t1Field, t2Field, cond...)
|
||||
}
|
||||
|
||||
func (q *selectQry) buildJoin(t Table, joinKW string, t1Field, t2Field Field, cond ...Conditioner) SelectClause {
|
||||
str := joinKW + " " + t.Name + " ON " + t1Field.String() + " = " + t2Field.String()
|
||||
if len(cond) == 0 { // Join with no condition
|
||||
q.join = append(q.join, str)
|
||||
return q
|
||||
}
|
||||
|
||||
func (q *selectQry) LeftJoin(t Table, t1Field, t2Field Field) SelectClause {
|
||||
q.join = append(q.join, "LEFT JOIN "+t.Name+" ON "+t1Field.String()+" = "+t2Field.String())
|
||||
return q
|
||||
// Join has condition(s)
|
||||
sb := getSB()
|
||||
defer putSB(sb)
|
||||
sb.Grow(len(str) * 2)
|
||||
|
||||
sb.WriteString(str + " AND ")
|
||||
|
||||
var argIdx int
|
||||
for i, c := range cond {
|
||||
argIdx = len(q.args)
|
||||
if i > 0 {
|
||||
sb.WriteString(" AND ")
|
||||
}
|
||||
sb.WriteString(c.Condition(&q.args, argIdx))
|
||||
}
|
||||
|
||||
func (q *selectQry) RightJoin(t Table, t1Field, t2Field Field) SelectClause {
|
||||
q.join = append(q.join, "RIGHT JOIN "+t.Name+" ON "+t1Field.String()+" = "+t2Field.String())
|
||||
return q
|
||||
}
|
||||
|
||||
func (q *selectQry) FullJoin(t Table, t1Field, t2Field Field) SelectClause {
|
||||
q.join = append(q.join, "FULL JOIN "+t.Name+" ON "+t1Field.String()+" = "+t2Field.String())
|
||||
q.join = append(q.join, sb.String())
|
||||
return q
|
||||
}
|
||||
|
||||
|
Reference in New Issue
Block a user