3. CSV Dumper + Fast API Access

Objective :
Periodically, dump content of (a really) big CSV file into a Redis cache, and access the data through public API.

Processes :
1. Download CSV source file, and this process run in the background and triggered periodically.
2. Periodically, read the downloaded CSV file, parse the data, and prepare them to be saved into Redis cache.
3. Saving the data into Redis.  As this process might content a huge number of "records" ,the saving process itself wrapped in the form of tasks (1 record 1 task), that will be passed into Exq queue. This task-of-saving will be excuted one by one, asynchronously.
4. Finally, once data available in Redis, the public API module will fetch them based on the API request parameter.

Source : https://github.com/dewa19/csv_dumper

Disclaimer :
1. Thanks to SpatialKey for letting me use their CSV sample file (Sample insurance portfolio) (https://support.spatialkey.com/spatialkey-sample-csv-data/)
2. The above scenario is developed as a part of learning process of understanding Elixir 

1. API Service with Redis

Objective :
Create a fully functional scenario of an API architecture using Redis as a cache mechanism. End user will have a benefit of fast response from in-memory database like Redis.

Internal (in house process) :
1. Read from 3rd party Provider (BART API, in this case)
2. Parse, save to database and Redis cache

Public API (exposed to end user) :
1. User API request
2. Read from Redis cache engine
3. If it can't find it in Redis, read from database (reply back) and save a copy into Redis for the next API requests.

Disclaimer :
1. 3rd party Data Provider, referring data provided by https://www.bart.gov/schedules/developers/api
2. The above scenario is developed as a process of learning Elixir