Web Scraping and Image Classification with TensorFlow

Web Scraping and Image Classification with TensorFlow
Web Scraping and Image Classification with TensorFlow

Web Scraping and Image Classification with TensorFlow

This project demonstrates my ability to combine web scraping techniques with machine learning to classify images. The goal was to build a pipeline that scrapes images of shoes from a website and uses them for training a machine learning model.

Key Features:

  • Web Scraping: I utilized BeautifulSoup and requests to scrape images from an e-commerce site, focusing on categories like formal shoes, sneakers, and boots. The images were automatically downloaded and saved into organized directories.
  • Image Preprocessing: The images were stored in separate folders for training and testing. This organization sets the foundation for an image classification task.
  • Model Setup: While the project focuses on scraping, the next logical step (which can be explored further) is to use TensorFlow and Keras to create a Convolutional Neural Network (CNN) model to classify these images into categories such as sneakers, formal shoes, and boots.
  • Scalability: The design of the scraping function allows it to easily scale for other shoe categories or even other types of products. Simply by adjusting the URLs and folder names, new categories can be added to the pipeline.

Technologies Used:

  • Python: The core language for web scraping and model building.
  • TensorFlow & Keras: For machine learning model development.
  • BeautifulSoup & Requests: For scraping data from the web.
  • PIL & Matplotlib: For handling images and visualizing results.

Why This Project?

This project not only helped me refine my web scraping skills but also gave me hands-on experience in integrating web data with machine learning models. It's an excellent example of how data from websites can be harvested and used in AI-driven applications, such as in e-commerce for product categorization.

💼 Business Impact

This project has the potential to significantly enhance business operations, especially for e-commerce platforms. By automating the categorization and classification of product images, businesses can save time and resources that would otherwise be spent on manual labor. Here are some key business impacts: Improved Product Categorization: Automated image classification ensures accurate categorization of products, which improves the shopping experience for customers. This makes it easier for businesses to organize their inventory and for customers to find what they are looking for quickly. Enhanced Customer Experience: By having accurate and consistent product categories, customers can easily navigate through various types of shoes (e.g., sneakers, formal shoes, boots), leading to improved satisfaction and higher conversion rates.

Code Snippet

import requests
from bs4 import BeautifulSoup
from PIL import Image
from IPython.display import display
import re
import os
from tensorflow import keras
from tensorflow.keras import layers
import matplotlib.pyplot as plt
import tensorflow as tf

# Load Data

def download_image(train_or_test,folder_name):
  all_url=[]
  res=requests.get(f'https://shoeplace.ng/10-formal-shoes').content
  be=BeautifulSoup(res,'lxml')
  for i in range(len(be.find_all('img'))):
    try:
      all_url.append(be.find_all('img')[i]['data-full-size-image-url'])
    except:
      pass


  for i in range(len(all_url)):
    r = requests.get(all_url[i])

    with open(f"shoes/{train_or_test}/{folder_name}/{i}.jpg",'wb') as f:

      f.write(r.content)
  return

  def download_image(start_page,end_page,item_name,folder_name,train_or_test):
  all_url=[]
  for i in range(start_page,end_page):
    res=requests.get(f'https://en.afew-store.com/collections/{item_name}?offset={i}').content
    be=BeautifulSoup(res,'lxml')

    for i in range(len(be.find_all('img'))):
      try:
        all_url.append(be.find_all('img')[i]['data-src'])
      except:
        pass


# webscraping
  for i in range(len(all_url)):
    r = requests.get('https:'+all_url[i])

    with open(f"shoes/{train_or_test}/{folder_name}/{i}.jpg",'wb') as f:

      f.write(r.content)
  return

# TensorFlow
  train=keras.utils.image_dataset_from_directory('/content/shoes/train',seed=seed,image_size=(180,180),validation_split=0.2,subset='training')
val=keras.utils.image_dataset_from_directory('/content/shoes/train',seed=seed,image_size=(180,180),validation_split=0.2,subset='validation')

model=keras.Sequential([
    layers.Rescaling(1./255),
    layers.Conv2D(60,5,activation='relu'),
    layers.MaxPool2D(),
    layers.Conv2D(80,6,activation='relu'),
    layers.MaxPool2D(),
    layers.Conv2D(100,8,activation='relu'),
    layers.MaxPool2D(),
    layers.Flatten(),
    layers.Dense(6)
])

Other Projects

Telco Churn
Telco Churn
Telco churn refers to the rate at which customers discontinue their services with a telecommunications company-whether it's mobile, internet, cable TV, or bundled services. It's a critical KPI in the telecom industry because high churn rates = revenue leakage + brand instability.
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
supply chain spend analytics dashboard overview
supply chain spend analytics dashboard overview
This project involved building an interactive Spend Analytics Dashboard to help logistics, procurement and finance teams gain real-time visibility into organizational spending. The dashboard consolidated multi-year spend data across vendors, product categories, and logistics channels - providing a centralized decision-support system for cost control and strategic sourcing.
Visualization
PowerBI
View Project
Scraping Airbnb
Scraping Airbnb
This project demonstrates my ability to combine web scraping techniques with machine learning to classify images. The goal was to build a pipeline that scrapes images of shoes from a website and uses them for training a machine learning model.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
Customers Transaction Analysis
Customers Transaction Analysis
A comprehensive database management system for a small business.
Database
SQL
MySQL
PostgreSQL
MongoDB
Pyspark
View Project
Bulldozer Sale Price Prediction
Bulldozer Sale Price Prediction
This project tackled a real-world business problem sourced from a Kaggle competition hosted by Caterpillar Inc. Using structured data containing product specifications, usage history, and transactional attributes, I built a machine learning pipeline capable of forecasting future equipment prices a key tool for optimizing resale strategy in capital-intensive industries.
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
Credit Score Prediction
Credit Score Prediction
This project focuses on building a predictive model to estimate an individual's credit score category-poor, Standard, or Good-based on financial and behavioral attributes. The goal is to assist financial institutions in automating credit risk assessment, improving loan approval decisions, and reducing default rates.
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
Customer Booking Prediction
Customer Booking Prediction
This Customer Booking Prediction project uses historical transaction data, customer behavior patterns, and contextual variables to forecast the likelihood and timing of future bookings. By applying machine learning models such as Logistic Regression, XGBoost, or Neural Networks, organizations can identify which customers are most likely to make a booking, when they are likely to do so, and what factors influence their decision
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
Bike Sharing Prediction
Bike Sharing Prediction
This project Bike Sharing Prediction leverages historical rental data, weather patterns, seasonal trends, and time-based variables to forecast future demand for shared bicycles. This predictive modeling process typically involves data preprocessing, feature engineering, and the application of machine learning algorithms such as Random Forest and Gradient Boosting
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
Customer Segmentation
Customer Segmentation
Developed a predictive classification model to determine the likelihood of a customer completing a booking. The project enables businesses to reduce cart abandonment, optimize marketing strategies, and increase conversion rates.
Data-Science
Python
Scikit-learn
Pandas
Numpy
Tensorflow
Keras
Pytorch
View Project
Sales Dashboard using Power BI
Sales Dashboard using Power BI
This Sales Performance Dashboard provides actionable insights into product sales, customer segmentation, and regional performance. Built with Power BI, this dashboard supports strategic decision-making by consolidating large volumes of transactional data into an interactive and visually intuitive interface.
Visualization
PowerBI
View Project
Vehicle Repair Dashboard
Vehicle Repair Dashboard
This Vehicle Repair Dashboard provides a strategic lens into fleet maintenance activities across suppliers, vehicle brands, and service types. It enables data-driven decision-making by highlighting repair patterns, supplier performance, and maintenance trends.
Visualization
PowerBI
View Project
British Airways Customer Review Web Scraping, Text Classification, Sentiment Analysis, and Customer Satisfaction
British Airways Customer Review Web Scraping, Text Classification, Sentiment Analysis, and Customer Satisfaction
This project demonstrates my ability to combine web scraping techniques with machine learning to classify images. The goal was to build a pipeline that scrapes images of shoes from a website and uses them for training a machine learning model.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
British Airway Webscraping with Beautifulsoup
British Airway Webscraping with Beautifulsoup
This project demonstrates how web scraping can be used to extract valuable data from airline websites, specifically British Airways, using Python's BeautifulSoup library. The goal was to scrape flight-related data, such as flight details, prices, and other useful information, from the British Airways website for analysis.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
Ecommerce Website Webscraping
Ecommerce Website Webscraping
This project uses requests and BeautifulSoup to scrape product listings from StockX’s e-commerce platform. The scraper extracts product name, description, price, and image link across multiple paginated results, storing them in a structured dataset for analysis or integration into business workflows.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
Jumia Website Scraping
Jumia Website Scraping
This project uses requests and BeautifulSoup to scrape product listings and prices from Jumia Nigeria. The scraper works across multiple pages and categories, returning structured datasets ready for analysis.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
Stack Overflow Text Classification
Stack Overflow Text Classification
This project involves building a multi-class classification model to predict the tag of a Stack Overflow question based on its content. The goal is to classify programming questions into specific tags such as Python, CSharp, JavaScript, and Java using Natural Language Processing (NLP) techniques.
Data-Science
Deep-Learning
Python
Numpy
Tensorflow
Keras
Pytorch
View Project
Fruits Image Classification
Fruits Image Classification
This project leverages image classification techniques to classify fruit images into their respective categories using Convolutional Neural Networks (CNN). The dataset used is the Fruits 360 dataset, which includes a wide variety of fruits and vegetables, and the goal was to build a model capable of predicting the type of fruit based on the image.
Data-Science
Deep-Learning
Python
Numpy
Tensorflow
Keras
Pytorch
View Project
Purchase Prediction
Purchase Prediction
Purchase Prediction with Logistic Regression.This project focuses on using logistic regression to predict customer purchases based on various demographic and behavioral features. The goal was to predict whether a customer will make a purchase (binary classification) using factors such as age, gender, marital status, income, occupation, and more.
Data-Science
Scikit-learn
Python
Numpy
K-Means
PCA
Logistic Regression
View Project
Ebay Sneakers WebScraping Using Selenium
Ebay Sneakers WebScraping Using Selenium
This project demonstrates the use of Python-based web scraping to extract product information from eBay. The focus was on collecting structured data such as item titles, prices, and condition from search result pages, enabling price analysis and market research.
WebScraping
BeautifulSoup
Selenium
Scrapy
View Project
Uber Ride Analytics Dashboard Overview
Uber Ride Analytics Dashboard Overview
This project involved building an interactive Uber Analytics Dashboard to provide insights into ride bookings, payment methods, vehicle usage, and location-based demand. The dashboard enabled stakeholders to monitor trends in customer behavior, evaluate revenue drivers, and optimize fleet allocation for operational efficiency and strategic planning.
Visualization
PowerBI
View Project
HR Analytics Dashboard Employee & Attrition Insights
HR Analytics Dashboard Employee & Attrition Insights
This project involved building an Interactive HR analytics dashboard covering employee attrition, hiring, salaries and departmental performance. Includes data visualizations and KPIs for strategic HR decision-making.
Visualization
PowerBI
View Project