1.4 KiB
ETL
Extract, transform and load DATA
We get data from multiple places and merge them into one place.
Definition of multiple places: - Retro Team Sharepoint - Future Osmosis Sharepoint
Definition of one place: - into a CSV...today (03/03/2025)
- Added sharepointclient that khalim made - Need to proof it works
- Read a file from what khalim has shared
Add a local file:
- mount a local folder directory wiht what Khalim sharepoint he has shared
- REad files and file path
Once I have sharepoint api working:
- [] Make validator for retro team
- [] once validated, produce a csv file
- [] show some cool productivity metric
Currently working on:
-
[On hold until i get sharepoint working] Validator
- check names
- [in progress, blocked unitl sharepoint. Easy to add] check it has dates
-
Useful file reader:
- Khalim showed me a useful pdf, that I should try to extract and get some information
-
[] Share point connection Figure out how to use the sharepoint connector
-
With Khalim:
-
Check if I have access to sharepoint
-
[] Try and get his client API working and see if I can read files
MVP: Script we can run that will Go to share point fetch all the data ( in progress ) provide some form of output that shows the number of surverys done (Get this information!!!)
Flat table
Billing: Billing table, left join