Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions .github/workflows/python-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ jobs:
with:
python-version: '3.13'

- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true

- name: Set up Java (required for Datastore emulator)
uses: actions/setup-java@v4
with:
Expand All @@ -44,14 +49,7 @@ jobs:
run: 'gcloud config set project python-datastore-sqlalchemy'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then
pip install -r requirements.txt
else
echo "No requirements.txt found"
fi
run: uv sync --locked

- name: Run tests
run: |
pytest
run: uv run pytest
65 changes: 49 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,65 @@
SQLAlchemy dialect for google cloud datastore(firestore mode)
SQLAlchemy dialect for Google Cloud Datastore in Firestore mode
========================
How to install
```
python3 setup.py install
```
or
```

## Installation

Install from PyPI:
```bash
pip install python-datastore-sqlalchemy
```
How to use

Install from a local checkout:
```bash
uv pip install -e .
```

Runtime dependencies are declared in `pyproject.toml`. `requirements.txt` is not used by this project.

## Usage

```python
from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
engine = create_engine("datastore://my-project/?database=my-db", credentials='path/to/credentials.json')
conn = engine.connect()
result = conn.execute("SELECT * fro" test_table")
print(result.fetchall())
from sqlalchemy import create_engine, text

engine = create_engine(
"datastore://my-project/?database=my-db",
credentials="path/to/credentials.json",
)

with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM test_table"))
print(result.fetchall())
```

## Preview

<img src="assets/pie.png">

> [!WARNING]
> Please note: Not all GQL and SQL syntax has been fully tested. Should you encounter any bugs, please post the relevant query and open an issue on GitHub.

## How to contribute

Feel free to open issues and pull requests on GitHub.

## Development

This project supports Python 3.10 and newer. It uses a uv-managed virtual environment by default. The lockfile is tracked so local development and CI use the same dependency resolution.

Install the development environment:
```bash
uv sync --locked
```

Run the test suite:
```bash
uv run pytest
```

The Datastore-backed tests require the Google Cloud SDK with the `beta` and `cloud-datastore-emulator` components installed. If `gcloud` is not on `PATH`, point the test fixture at it explicitly:
```bash
GCLOUD_PATH=/path/to/google-cloud-sdk/bin/gcloud uv run pytest
```

## Development Notes
- [Develop a SQLAlchemy and it's dialects](https://hackmd.io/lsBW5GCVR82SORyWZ1cssA?view)

- [Develop a SQLAlchemy and its dialects](https://hackmd.io/lsBW5GCVR82SORyWZ1cssA?view)
67 changes: 67 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,70 @@
[build-system]
requires = ["setuptools>=64"]
build-backend = "setuptools.build_meta"

[project]
name = "python-datastore-sqlalchemy"
version = "9.0.3"
description = "SQLAlchemy dialect for google cloud datastore"
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
{ name = "HY Chang(splasky)", email = "hychang.1997.tw@gmail.com" },
]
classifiers = [
"Development Status :: 2 - Pre-Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: Implementation :: CPython",
"Topic :: Database :: Front-Ends",
"Operating System :: POSIX :: Linux",
]
keywords = ["SQLAlchemy", "GCP", "Datastore"]
dependencies = [
"sqlalchemy>=1.4.16,<3.0.0",
"google-cloud-datastore>=2.21.0",
"google-cloud-firestore>=2.21.0",
"google-cloud-bigquery>=3.35.0",
"google-auth>=2.40.0",
"sqlglot>=28.0.0",
"pandas>=2.0.0",
"requests",
]

[project.entry-points."sqlalchemy.dialects"]
datastore = "sqlalchemy_datastore:CloudDatastoreDialect"

[project.urls]
Documentation = "https://github.com/splasky/python-datastore-sqlalchemy/wiki"
Source = "https://github.com/splasky/python-datastore-sqlalchemy"
Tracker = "https://github.com/splasky/python-datastore-sqlalchemy/issues"

[dependency-groups]
dev = [
"coverage==7.9.2",
"flake8==7.3.0",
"mypy",
"pytest==9.0.3",
"pytest-cov==6.2.1",
"pytest-html==4.1.1",
"pytest-mypy==1.0.1",
"pytest-ruff==0.5",
"ruff",
]



[tool.setuptools.packages.find]
include = ["sqlalchemy_datastore*"]

[tool.pytest.ini_options]
addopts = "--ruff --mypy --cov --cov-report=html"
pythonpath = ["."]
testpaths = ["tests"]
python_files = ["test_*.py"]

[tool.ruff]
line-length = 120

Expand Down
24 changes: 0 additions & 24 deletions pytest.ini

This file was deleted.

15 changes: 0 additions & 15 deletions requirements.txt

This file was deleted.

62 changes: 2 additions & 60 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,64 +17,6 @@
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

import os
import re
from setuptools import setup

from setuptools import setup, find_packages

with open(
os.path.join(os.path.dirname(__file__), "sqlalchemy_datastore", "__init__.py")
) as v:
version_match = re.compile(r'.*__version__ = "(.*?)"', re.S).match(v.read())
if not version_match:
raise RuntimeError("Unable to find version string in __init__.py.")
VERSION = version_match.group(1)

readme = os.path.join(os.path.dirname(__file__), "README.md")
with open(readme) as f:
long_description = f.read()


setup(
name="python-datastore-sqlalchemy",
version=VERSION,
description="SQLAlchemy dialect for google cloud datastore",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/splasky/python-datastore-sqlalchemy",
author="HY Chang(splasky)",
author_email="hychang.1997.tw@gmail.com",
license="MIT",
classifiers=[
"Development Status :: 2 - Pre-Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: Implementation :: CPython",
"Topic :: Database :: Front-Ends",
"Operating System :: POSIX :: Linux",
],
keywords="SQLAlchemy GCP Datastore",
python_requires=">=3.9",
project_urls={
"Documentation": "https://github.com/splasky/python-datastore-sqlalchemy/wiki",
"Source": "https://github.com/splasky/python-datastore-sqlalchemy",
"Tracker": "https://github.com/splasky/python-datastore-sqlalchemy/issues",
},
packages=find_packages(include=["sqlalchemy_datastore"]),
include_package_data=True,
install_requires=[
"sqlalchemy>=1.4.16,<3.0.0",
"google-cloud-datastore>=2.21.0",
"google-cloud-firestore>=2.21.0",
"google-cloud-bigquery>=3.35.0",
"google-auth>=2.40.0",
"sqlglot>=28.0.0",
"pandas>=2.0.0",
"requests",
],
zip_safe=False,
entry_points={
"sqlalchemy.dialects": ["datastore = sqlalchemy_datastore:CloudDatastoreDialect"]
},
)
setup()
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ def datastore_client():
"start",
"--host-port=localhost:8081",
"--no-store-on-disk",
"--consistency=1.0",
"--quiet",
]
)
Expand Down
Loading
Loading