Use Case
Not everyone may need it, but I encountered a problem where I wanted to gather detailed information about my competitors’ (in the e-commerce industry) website sales and new product releases. However, I didn’t want to be glued to my computer like I would be for futures or forex trading, constantly monitoring their website.
So, I thought of using a script to do the job. The script’s functionality would be to notify me through email if a competitor sells a product, providing the name and price of the sold item. It would also notify me about newly launched products, including their names and prices. This would be helpful for analyzing my competitors’ sales volume and trends and making informed decisions about which products to sell in my own store.
Additionally, this script can be modified for various other purposes, such as monitoring the rise of your TikTok followers, tracking the progress of other events or news, or even staying updated on current hot topics.
Existing Solutions in the Market
Of course, the idea I mentioned earlier is applicable to every company or store in reality, and there are dedicated programs available for this purpose. Some good website monitoring tools include Visualping, Distill Web Monitor, and Wachete, among others. These tools are professional and well-developed, but they often come with a cost if you want to use them extensively.
There are also similar free products available, and I have tried them. However, they have limitations, such as allowing only 10 email notifications for a maximum of one website change per day. If you exceed this limit, you’ll have to pay for additional services. It’s simply not sufficient!
So, why not write your own code and use it?
The Code You Can Write Yourself
import requests
from bs4 import BeautifulSoup
import smtplib
from email.mime.text import MIMEText
import time
# Website URL
url = "https://"
# Email parameters
sender = 'your_sender_email'
receiver = 'your_receiver_email'
smtp_server = 'smtp.xxx.com' # SMTP server address of the sender's email
smtp_port = 465 # SMTP port of the sender's email
username = 'your_sender_email'
password = 'your_sender_email_password'
def send_email(subject, body):
# Create MIMEText email
msg = MIMEText(body)
msg['Subject'] = subject
msg['From'] = sender
msg['To'] = receiver
# Send email
with smtplib.SMTP_SSL(smtp_server, smtp_port) as server:
server.login(username, password)
server.sendmail(sender, receiver, msg.as_string())
def get_product_info(product_url):
# Get product information
response = requests.get(product_url)
soup = BeautifulSoup(response.text, 'html.parser')
title = soup.find('h1', class_='h3').text
price = soup.find('span', class_='h2').text
stock = soup.find('span', class_='js-product-stock').text
return (title, price, stock)
def check_product_sold_out(product_url):
# Check if the product is sold out
response = requests.get(product_url)
soup = BeautifulSoup(response.text, 'html.parser')
if soup.find('div', class_='product-sold-out'):
return True
else:
return False
# Regularly check the products
while True:
# Get the webpage content
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# Find the product list
product_list = soup.find('div', class_='js-product-list')
# Check if each product is sold out
for product in product_list.find_all('a', class_='product-card'):
product_url = product['href']
product_title = product.find('h2').text
if check_product_sold_out(product_url):
# The product is sold out, send an email notification
subject = f'Product Sold Out: {product_title}'
body = f'Product Name: {product_title}\n'
send_email(subject, body)
else:
# The product is not sold out, check if it's a new product
try:
# Try to get the product information, if it fails, it means it's a new product
product_title, product_price, product_stock = get_product_info(product_url)
except:
# It's a new product, send an email notification
subject = f'New Product Released: {product_title}'
body = f'Product Name: {product_title}\nStock: {product_stock}\nPrice: {product_price}\n'
send_email(subject, body)
# Wait for a certain period of time before checking again
time.sleep(300)Explanation: This is a monitoring program for web stores. Once there’s a new product release or a product is sold, the program will promptly notify you, just like a spy!
Here’s an additional gift
import requests
import hashlib
import time
import smtplib
from email.mime.text import MIMEText
url = 'https://'
def get_hash(url):
response = requests.get(url)
return hashlib.sha256(response.content).hexdigest()
def send_email(content):
sender = ''
receiver = ''
password = ''
smtp_server = ''
smtp_port = 465
message = MIMEText(content)
message['From'] = sender
message['To'] = receiver
message['Subject'] = 'Website Change Alert'
server = smtplib.SMTP_SSL(smtp_server, smtp_port)
server.login(sender, password)
server.sendmail(sender, receiver, message.as_string())
server.quit()
current_hash = get_hash(url)
while True:
new_hash = get_hash(url)
if new_hash != current_hash:
send_email('Website content has changed.')
current_hash = new_hash
else:
time.sleep(30)Explanation: This code is simple. It compares webpage hashes, and whenever there’s a change, it sends an email notification. It can be used in any scenario!
How to Use This Code
nohup python3 web_monitoring.py > output.log 2>&1 &
The code provided above is in Python, and once you make the necessary modifications, you can use it directly. Here’s a method to run it in the background:
You can set up a Python environment on your VPS and follow the instructions mentioned above to run the code automatically. It’s quite convenient, isn’t it?”
