diff --git a/modules/ml-pipeline/.gitignore b/modules/ml-pipeline/.gitignore index 435bf5b..a4e6cb7 100644 --- a/modules/ml-pipeline/.gitignore +++ b/modules/ml-pipeline/.gitignore @@ -4,3 +4,8 @@ __pycache__/ .DS_Store .vscode/ data/ +/.DS_Store/ +*/.ipynb_checkpoints/ +*/.idea/ +*/.pytest_cache/ +*/.tox/ diff --git a/modules/ml-pipeline/README.MD b/modules/ml-pipeline/README.MD index 2711569..38f5bb9 100644 --- a/modules/ml-pipeline/README.MD +++ b/modules/ml-pipeline/README.MD @@ -18,13 +18,13 @@ Within `src` folder, the structure is as follows: # How to develop using this pipeline: Run `make init`, which will: -- Download pyenv (Python version management) +- Download miniconda (Conda version management) - Download Python 3.X.X as defined in the `make` file - current 3.10.12 - Create a virtual environment with this version of python - Install packages in the training and version control directories in the pipeline folder (dev version if applicable) - Install pre-commit to enable pre-commit hooks -To use the environment, run `source .dev_env_pipeline/bin/activate`. +To use the environment, run `conda activate dev_env_pipeline` To enable the virtual envrionemnt created in vscode: - Open settings @@ -41,11 +41,11 @@ To use the docker environment for coding in VSCODE: - Add the Git extension (for dvc) For running experiment, everything will be cached but the workflow will be: -- `dvc repro` to regenerate the current experiement +- `dvc repro` to regenerate the current experiement - Change parameters if needed - Use `dvc exp run` - Cachec the results by using `dvc push -r REMOTE_NAME` - Repeat as needed - When happy with results, use `dvc exp apply EXPERIMENT_NAME` -- Use `dvc pull` -- Commit code \ No newline at end of file +- Use `dvc pull` +- Commit code