ETL

Extract, transform and load DATA

We get data from multiple places and merge them into one place.

Definition of multiple places:
    - Retro Team Sharepoint
    - Future Osmosis Sharepoint

Definition of one place:
    - into a CSV...today (03/03/2025)

- [x] Added sharepointclient that khalim made - Need to proof it works
- [x] Read a file from what khalim has shared

Add a local file:
- [x] mount a local folder directory wiht what Khalim sharepoint he has shared
- [x] REad files and file path


Once I have sharepoint api working:
- [] Make validator for retro team
- [] once validated, produce a csv file
- [] show some cool productivity metric

Currently working on:
- [On hold until i get sharepoint working] Validator
    - [x] check names
    - [in progress, blocked unitl sharepoint. Easy to add] check it has dates
- [x] Useful file reader:
    - [x] Khalim showed me a useful pdf, that I should try to extract and get some information
- [] Share point connection
        Figure out how to use the sharepoint connector

- With Khalim:
- [x] Check if I have access to sharepoint
- [] Try and get his client API working and see if I can read files


MVP:
 Script we can run that will
 Go to share point fetch all the data ( in progress )
 provide some form of output
 that shows the number of surverys done (Get this information!!!)

Flat table
<Survery name> <cUSTOMER NAME> <DETAILS> <installer>

Billing:
Billing table, left join