Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Anonymization Service

SDV based tabular data anonymization tool with a 4-step workflow:

  1. Upload data (CSV / XLS / XLSX) 1_Dataupload
  2. Detect PII 2_Scan
  3. Configure rules (exempt_columns / force_pii_columns) 3_Config
  4. Generate, preview, compare, and download synthetic data in csv format 4_result

Project Structure

  • src/
    • api.py - FastAPI endpoints
    • generate_metada.py - metadata generation and config save
    • Gaussian_Coupla_syntheticdata_generator.py - SDV synthetic data pipeline
  • ui/ - frontend page and step scripts
  • results/ - runtime outputs
    • uploaded_data/ - uploaded files converted to CSV
    • meta_data/ - generated metadata and user config
    • GaussianCopula_results/ - synthetic data, quality reports, scores
  • raw_data/ - optional mounted input folder (Docker use)

Requirements

  • Python 3.9+
  • Dependencies in requirements-api.txt
  • spaCy model:
    • python -m spacy download en_core_web_sm

Install through Docker

1) Prerequisites

2) Run

git clone https://github.com/gyt197/Data-Anonymization-Service.git
cd Data-Anonymization-Service
docker compose up --build

3) Access

App: http://localhost:8000

4) Stop

docker compose down

Docker Setup

Install from Github

1) Clone the repository

git clone https://github.com/gyt197/Data-Anonymization-Service.git

2) Navigate into the project

cd Data-Anonymization-Service

3) Create a virtual environment

python -m venv venv

4) Activate the virtual environment

On Windows

my-env\Scripts\activate

5) Install dependencies

pip install -r requirements-api.txt
pip install sdv python-multipart
python -m spacy download en_core_web_sm

6) Run the server

uvicorn src.api:app --host 0.0.0.0 --port 8000 --reload

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages