mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-06-30 13:10:47 +00:00
commit
5afbb6ed7b
2 changed files with 19 additions and 12 deletions
1
.github/workflows/deploy_terraform.yml
vendored
1
.github/workflows/deploy_terraform.yml
vendored
|
|
@ -316,6 +316,7 @@ jobs:
|
||||||
stage: ${{ needs.determine_stage.outputs.stage }}
|
stage: ${{ needs.determine_stage.outputs.stage }}
|
||||||
ecr_repo: postcode_splitter-${{ needs.determine_stage.outputs.stage }}
|
ecr_repo: postcode_splitter-${{ needs.determine_stage.outputs.stage }}
|
||||||
image_digest: ${{ needs.ordnanceSurvey_image.outputs.image_digest }}
|
image_digest: ${{ needs.ordnanceSurvey_image.outputs.image_digest }}
|
||||||
|
terraform_apply: ${{ needs.determine_stage.outputs.terraform_apply }}
|
||||||
secrets:
|
secrets:
|
||||||
AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
||||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
||||||
|
|
|
||||||
|
|
@ -1,20 +1,26 @@
|
||||||
We have list of address as input.
|
So you want to fetch UPRN for an address list?
|
||||||
|
|
||||||
It'll come in batches of the same post code and from then we want to somehow convert that into UPRN
|
|
||||||
|
|
||||||
if this lambda/function can do that we'll be speeding ahead
|
|
||||||
|
|
||||||
|
|
||||||
Energy Performance Information: https://epc.opendatacommunities.org/
|
Before you run:
|
||||||
|
|
||||||
guidance page: https://epc.opendatacommunities.org/docs/guidance#field_domestic_LMK_KEY
|
Step 1) Get the list and ensure the following columns exists
|
||||||
|
|
||||||
Example of past khalims code that he wrote some tests for: https://github.com/Hestia-Homes/Model/blob/941be42b83a590e838fd3ee475bfd1ff31438789/backend/tests/test_search_epc.py#L11
|
* Address 1
|
||||||
|
* Address 2
|
||||||
|
* Address 3
|
||||||
|
* postcode
|
||||||
|
|
||||||
|
And save it as a .csv file
|
||||||
|
|
||||||
|
|
||||||
Example of EPC search: https://github.com/Hestia-Homes/Model/blob/941be42b83a590e838fd3ee475bfd1ff31438789/backend/SearchEpc.py#L118
|
Step 2)
|
||||||
|
|
||||||
|
Before we run this, we need to upload it into S3 as well as put initiate a subtask + task
|
||||||
|
|
||||||
|
* S3 upload I'll recommend somewhere in retrofit-data-dev and get the s3_uri
|
||||||
|
|
||||||
|
For this example I'll be using "s3://retrofit-data-dev/ara_raw_inputs/calico/formated(Sheet1).csv"
|
||||||
|
|
||||||
|
Go to Ara DB and make a new task_id with a randomly generated uuid as the primarily key
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Khalim has made a python package to help scrape data: https://github.com/KhalimCK/epc-api-python
|
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue