Disclaimer

Disclaimer

This is my personal notes on Erlang & Elixir that serve as Backend Services. This  note is the way I remembering everything I've lea...

Monday, July 8, 2019

3. CSV Dumper + Fast API Access








Objective :
Periodically, dump content of (a really) big CSV file into a Redis cache, and access the data through public API.


Processes :
1. Download CSV source file, and this process run in the background and triggered periodically.
2. Periodically, read the downloaded CSV file, parse the data, and prepare them to be saved into Redis cache.
3. Saving the data into Redis.  As this process might content a huge number of "records" ,the saving process itself wrapped in the form of tasks (1 record 1 task), that will be passed into Exq queue. This task-of-saving will be excuted one by one, asynchronously.
4. Finally, once data available in Redis, the public API module will fetch them based on the API request parameter.


Source : https://github.com/dewa19/csv_dumper


Disclaimer :
1. Thanks to SpatialKey for letting me use their CSV sample file (Sample insurance portfolio) (https://support.spatialkey.com/spatialkey-sample-csv-data/)
2. The above scenario is developed as a part of learning process of understanding Elixir 


2. Lookup_Email : An Email Validator





Objective :
The objective of this project is to validate an email address from the side of SMTP server, without really sending a message to given email. If -by sending a series of SMTP commands- an email is acknowledged by its SMTP server, then it is an "valid" email.

Processes :
1. Fetch domain name of the email
2. Verify the domain
3. If its verified, send some SMTP commands
4. Based on its responses, we take an "educated guess" about the status of a given email


Source : https://github.com/dewa19/lookup_email