Core Python Development Expert with a focus on automation, web scraping, data engineering and generative AI. Experienced in developing ETL/ELT pipelin... Read More
Skills
LangChain
Ollama
Async
OpenAI
MySQL
Snowflake
PySpark
Experiences
6 years 2 months
Sequentum
May 2023 - May 2024
Python Lead
I am focused in this role on company-wide any kind of python automation work, mentoring colleagues & seniors in learning python and implementing it in their day-to-day data analysis assigned tasks.
Worked on JIRA, Confluence Pages for the Project Management and timely deliveries of tasks assigned to me.
Implemented Docker/Python Based concurrent Web Scrappers for some websites to overcoming the captcha blocking issues effectively.
Led Python/C# Big Data OOPS based China Bots Migration Project to Sequentum Enterprise Software.
Mentored my teammates to effectively understand C# and Python Code for better implementation in SE Software.
Integrated Node.js, python, C# Code effectively in Sequentum Enterprise.
Worked on Pyspark/Pyarrow based Parquet Files Automation, snowflake Data Loading/Unloading Automation.
Learnt Frida Toolkit to make one BOT work along with SE.
Working Experience with Databricks.
Used Fiddler/Burpsuite/Charles Proxy/Firefox Dev Tools For monitoring of Web Traffic/Android Apps Traffic for finding of API Requests those data was required to client.
Learning FRIDA in More Depth.
Developed Pyspark, Hadoop Based S3 Data Transformation Automation for Deduplicating, getting missing inputs, running queries on complete prefix data locally.
Worked on MYSQLWorkbench, SSMS Tools for Database Management/Manipulation of SE Software Bot Extracted DataSet.
Sequentum
Jul 2021 - May 2023
Senior Python Developer
Involved in Automation of Big DATA verification/data cleansing before delivering it to the clients using Python, PySpark, SPARKSQL, SNOWSQL.
Automated complex business processes transformations using pandas, Numpy, Dark Data Processing Frameworks.
Automated Complex System Processes like Windows OS Automation, AWS S3 Automation, Snowflake Data Warehouse Automation, Browser Automation, GUI Automation.
Sequentum
Dec 2020 - Jul 2021
Software Developer
Worked on Sequentum Enterprise(Content Grabber) software.
Automated mutiple business processes
Worked on big data technologies.
Worked on various Captcha Bypassing using Python such as Cloudflare Bypass, Google Recaptcha Bypass, Imperva.
scholarsbook
Aug 2019 - Aug 2020
Python Developer
Responsible for providing quality data for learners that comes from a variety of resources .
Responsible for complete data processing from Extracting to Deployment.
Developed and maintained web crawlers and data scrappers using Test Driven Development.
Gained extensive experience in loading extracted data from various resources directly to cloud(AWS) database in JSON format.
Developed and maintained ETL pipelines for daily ingestion of large data set into cloud database.
Extensively Worked with python on AWS EC2. Have extensive experiences in AWS E2 Issues Debugging & Troubleshooting.
Developed REST API's with FLASK and SQL Alchemy ORM.
Worked with latest data scrapping strategies to beat anti-bot detection technologies Implemented by Facebook , LinkedIn , Instagram to stop spiders to crawl their users data.
Learned Apache Kafka, Spark and Airflow.
Empirical Solutions
Mar 2018 - May 2019
Python Developer
Developed Rest APIs using Flask, MongoDB.
Developed Web Extraction Scripts in Python for Input Data Ingestion in Multiple Web Applications Cloud Databases such as AWS RDS.
Developed Single Page Applications using React.
Integrated React SPA WebApps in Angular 7 Web Applications.
Learnt TypeScript Programming Language.
Portfolio
No Portfolio added yet
Download the medial app to read full posts, comements and news.
Know everything thatโs happening in the startup ecosystem, first.