Principal responsibilities
- Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control centre.
- Provide expertise and hands on experience working on Kafka connect using schema registry in a very high volume.
- Provide administration and operations of the Kafka platform - provisioning, access lists and SSL configurations.
- Provide expertise and hands on experience working on Kafka connectors such as MQ connectors, Elastic Search connectors, JDBC connectors, File stream connector, JMS source connectors, Tasks, Workers, converters, Transforms.
- Provide expertise and hands on experience on custom connectors using the Kafka core concepts and API.
- Create topics, setup redundancy clusters, deploy monitoring tools, and configure appropriate alerts
- Create stubs for producers, consumers, and consumer groups for helping onboard applications from different languages/platforms.
- Automate routine tasks using scripts or automation tools.
- Perform data related benchmarking, performance analysis and tuning.
- Conduct root cause analysis of production incidents, documenting for reference and initiating proactive measures to enhance system reliability.
Qualifications
- Must be able to communicate on technical levels with Engineers and stakeholders.
- Strong problem solving and analytical skills
- Thorough understanding of the Kafka Architecture
- Familiar with cluster maintenance processes and implementing changes and recommended fixes to Kafka clusters and topics to protect production.
- Experience operating in an infrastructure as code and automation first principles environment
What additional skills will be good to have?
- Kubernetes knowledge
- Messaging technologies –Apache Kafka, Confluent Kafka
- DevOps toolsets – GitHub, JIRA, Confluence, Jenkins
- Automation – Ansible, Puppet or similar
- Monitoring –Observability tools such as Prometheus, Grafana, Splunk
