3255 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Daniel Roth	df6c42933a	do the database write	2026-02-10 09:59:23 +00:00
Daniel Roth	da34b99251	tidy up logs further	2026-02-10 09:43:55 +00:00
Daniel Roth	d4b444f49b	Fix bad log	2026-02-10 09:43:18 +00:00
Daniel Roth	d0847681c1	tidy up logs	2026-02-10 09:42:29 +00:00
Daniel Roth	06539a787d	use streaming mode in openpyxl for peabody parser	2026-02-10 09:30:39 +00:00
Daniel Roth	c0534d9546	more logs	2026-02-10 09:16:52 +00:00
Daniel Roth	b026a7ae9e	log db session but dont write to db	2026-02-10 09:02:44 +00:00
Daniel Roth	cd0ae37d40	more logging	2026-02-09 17:18:45 +00:00
Daniel Roth	9b6f99929e	add logging	2026-02-09 17:00:59 +00:00
Daniel Roth	3fa51e3c3a	extend lambda timeout to 3 minutes	2026-02-09 16:32:25 +00:00
Daniel Roth	d1e9e9f7ab	uncomment if: env.STAGE == 'prod'	2026-02-09 16:10:00 +00:00
Daniel Roth	1f1e077b49	use var.stage when defining terraform state s3 key	2026-02-09 16:05:42 +00:00
Daniel Roth	888d46f275	correct tfstate bucket and path	2026-02-09 15:52:15 +00:00
Daniel Roth	a1bb768dfe	correct path to tfstate file	2026-02-09 15:20:06 +00:00
Daniel Roth	de6bb935fe	correct role import path in condition etl tf	2026-02-09 15:10:29 +00:00
Daniel Roth	7630067d8d	expose role name from lambda_with_sqs	2026-02-09 15:05:01 +00:00
Daniel Roth	658761933b	read policy ARN from shared stack state	2026-02-09 14:56:46 +00:00
Daniel Roth	1e1db67f3c	import shared module into condition-etl tf	2026-02-09 14:34:29 +00:00
Daniel Roth	dfff8adc3d	Apply: generate new role for reading from s3 bucket and attach it to the lambda	2026-02-09 14:20:22 +00:00
Daniel Roth	f3e77beefd	generate new role for reading from s3 bucket and attach it to the lambda	2026-02-09 14:13:06 +00:00
Daniel Roth	68b12c7344	trigger rebuild and deploy	2026-02-09 12:49:32 +00:00
Daniel Roth	db0431f413	recomment env.stage == 'prod'	2026-02-09 12:20:44 +00:00
Daniel Roth	2e424e227e	apply shared stack to generate new s3 bucket	2026-02-09 12:19:17 +00:00
Daniel Roth	56e78350fa	configure allowed_origins	2026-02-09 12:04:24 +00:00
Daniel Roth	a2cdf0eb57	fix bucketname typo	2026-02-09 12:02:13 +00:00
Daniel Roth	092d95fa9d	create new s3 bucket for condition data	2026-02-09 11:45:59 +00:00
Daniel Roth	e1835b2cfa	remove duplicates from .env.local	2026-02-09 10:16:18 +00:00
Daniel Roth	0e4da14673	get lookup file bucket and key from payload per message	2026-02-09 09:56:41 +00:00
Daniel Roth	6dbb8149fd	reinstate used github secrets	2026-02-09 09:37:43 +00:00
Daniel Roth	9a9ea9fbaa	remove unused bits of config	2026-02-09 09:36:41 +00:00
Daniel Roth	f41da9e93f	remove commented out deployment jobs	2026-02-09 09:28:17 +00:00
Daniel Roth	daaa3c560f	revert changes to _deploy_lambda	2026-02-09 09:20:50 +00:00
Daniel Roth	bae1735e23	try using shared resource to fetch secrets	2026-02-06 16:22:53 +00:00
Daniel Roth	f033b144c2	including environment in vars not environment_vars	2026-02-06 16:04:57 +00:00
Daniel Roth	d44fdcd01c	try again	2026-02-06 15:22:36 +00:00
Daniel Roth	2fe9e67ea9	ensure env vars string is correct format	2026-02-06 15:16:15 +00:00
Daniel Roth	4c0713b415	include env vars in condition-etl lambda variabls	2026-02-06 15:06:46 +00:00
Daniel Roth	78caa1e6ae	format env vars correctly in terraform plan	2026-02-06 14:49:53 +00:00
Daniel Roth	aff3113d52	only load env vars to lambda during deployment if they are provided	2026-02-06 14:44:50 +00:00
Daniel Roth	dda05a4f33	include environment vars in lambda terraform plan	2026-02-06 14:34:43 +00:00
Daniel Roth	807d94ca3c	test again with limited imports	2026-02-06 13:34:08 +00:00
Daniel Roth	948c238468	include environment_vars on deploy_lambda workflow	2026-02-06 13:25:58 +00:00
Daniel Roth	321a4505fb	no need to get region as env var	2026-02-06 13:23:15 +00:00
Daniel Roth	f66aa25268	new terraform job to fetch db credentials from secrets manager	2026-02-06 13:19:36 +00:00
Daniel Roth	04bc8a1407	include psycopg2	2026-02-06 11:51:31 +00:00
Daniel Roth	3445060ef2	check db user value	2026-02-06 11:46:58 +00:00
Daniel Roth	9ae81c1099	reinstate build args and check whether imports work now	2026-02-06 11:39:53 +00:00
Daniel Roth	f10951f81d	try without build args	2026-02-06 11:24:00 +00:00
Daniel Roth	b8283abc45	try getting all db vars from github secrets	2026-02-06 11:16:34 +00:00
Daniel Roth	9c7678865f	add env to step	2026-02-05 17:32:32 +00:00