Example
1. Monitor error logs using Log4j to debug and fix problems.
Description: A logging framework is important for any Java/J2EE based application. By changing the configuration, you can easily write your logs to different places. You can write your Java application logs to a database, files on the disk, a log management system, a console, Syslog or possibly other places in the network without changing your application code
Source: https://stackify.com/log4j-guide-dotnet-logging/
My requirement:
1.Performing transformations over the data using spark and AWS Elastics Map Reduce(EMR) according to the business requirement
Description – ?
Source – ?
2.Automated Oozie workflow upon EMR Cluster spinup to ETL Brokerage Data to Hive/Data Lake
Description – ?
Source – ?
3.Job automation with Ctrl-M to spin up EMR Cluster when Mainframe Data file landed on-prem
Description – ?
Source – ?
4.Utilized Atlassian Bitbucket and Bamboo for Code repository, reviews, builds, and deployments to AWS
Description – ?
Source – ?
5.Troposphere and CloudFormation template for handling S3 Bucket’s and EMR Cluster spinup’s
Description – ?
Source – ?
6.Applying security policies using AWS IAM(Identity and Access Management) to control access to the data in cloud
Description – ?
Source – ?
7.Utilized Splunk to pick up logs from EMR/CloudWatch for email alerts on Brokerage Data’s ETL status
Description – ?
Source – ?
8.Elevating code into the development, test, and production environments on schedule. Providing follow up Production support. Submitting change control requests and documents
Description – ?
Source – ?
9.Coordinating with Data tier for system integration and end-to-end development
Description – ?
Source – ?
10.Created Performance Testing program in Python Spark to compare between two big data result sets
Description – ?
Source – ?
11.Working with multiple Data analytics teams to provide enriched data to them so that it will be used for further analysis and reporting
Description – ?
Source – ?
12.Tracking code quality through SonarQube
Description – ?
Source – ?
13.Continuous Integration with Bamboo, Jenkins, and Maven
Description – ?
Source – ?
14.Monitoring and improvising the end-user experience in using query tools
Description – ?
Source – ?
