We are looking for a Software Development Engineer to build and maintain high throughput, reliable, performant, and cost-effective data collection and extraction modules using Node.js and Python. The role involves deploying, maintaining, and supporting these modules on AWS and GCP cloud platforms. You will be responsible for indexing, archiving, and retaining necessary data in multiple persistence stores such as Object stores (S3), Key value store (Dynamo DB), and Elastic Search based on the use case. Additionally, you will manage the quality of data collected using data quality libraries built with SQL/Python/Spark on AWS Glue, which will be exposed as dashboards for monitoring using AWS QuickSight and Kibana. The position requires you to abstract the collected data to downstream applications through a Node.js backend and collaborate with engineers, researchers, and data implementation specialists to design and create advanced, elegant, and efficient end-to-end competitive intelligence solutions.