Prerequisites
- Basic understanding of programming concepts ๐
- Python installation (3.8+) ๐
- VS Code or preferred IDE ๐ป
What you'll learn
- Understand the concept fundamentals ๐ฏ
- Apply the concept in real projects ๐๏ธ
- Debug common issues ๐
- Write clean, Pythonic code โจ
๐ฏ Introduction
Welcome to the exciting world of fuzz testing! ๐ Have you ever wondered how to find those sneaky bugs that only appear with weird, unexpected inputs? Thatโs where fuzz testing comes to the rescue!
In this tutorial, weโll explore how to use random input generation to stress-test your Python code and uncover hidden bugs. Whether youโre building web APIs ๐, command-line tools ๐ฅ๏ธ, or data processing pipelines ๐, fuzz testing will help you build more robust software.
By the end of this tutorial, youโll be generating random test data like a pro and catching bugs before your users do! Letโs dive in! ๐โโ๏ธ
๐ Understanding Fuzz Testing
๐ค What is Fuzz Testing?
Fuzz testing is like hiring a mischievous gremlin ๐น to throw random garbage at your code until something breaks! Think of it as a stress test where instead of carefully crafted test cases, you generate thousands of random inputs to see what happens.
In Python terms, fuzz testing means:
- โจ Automatically generating random test inputs
- ๐ Running your code with these inputs thousands of times
- ๐ก๏ธ Catching edge cases you never thought about
๐ก Why Use Fuzz Testing?
Hereโs why developers love fuzz testing:
- Finds Hidden Bugs ๐: Discovers edge cases humans miss
- Saves Time โฐ: Automated testing runs while you sleep
- Improves Security ๐: Catches potential security vulnerabilities
- Builds Confidence ๐ช: Know your code handles anything
Real-world example: Imagine testing a password validator ๐. With fuzz testing, you might discover it crashes when given Unicode emojis or extremely long strings!
๐ง Basic Syntax and Usage
๐ Simple Random Input Generation
Letโs start with basic random input generation:
import random
import string
# ๐ Hello, Fuzz Testing!
def generate_random_string(length=10):
"""Generate a random string with letters and digits ๐ฒ"""
chars = string.ascii_letters + string.digits
return ''.join(random.choice(chars) for _ in range(length))
# ๐จ Generate different types of random data
def generate_random_inputs():
return {
'string': generate_random_string(),
'integer': random.randint(-1000, 1000),
'float': random.uniform(-100.0, 100.0),
'boolean': random.choice([True, False]),
'emoji': random.choice(['๐', '๐', '๐ฅ', '๐'])
}
# ๐ฎ Let's see what we get!
for _ in range(3):
print(f"Random inputs: {generate_random_inputs()}")
๐ก Explanation: Weโre using Pythonโs random
module to generate different types of test data. The emoji in the output makes debugging more fun!
๐ฏ Using the hypothesis Library
Python has an amazing library called hypothesis
for property-based testing:
from hypothesis import given, strategies as st
import hypothesis
# ๐ฏ Define a function to test
def divide_numbers(a, b):
"""Divide two numbers safely ๐ข"""
if b == 0:
return None
return a / b
# ๐งช Fuzz test with hypothesis
@given(
a=st.floats(min_value=-1000, max_value=1000),
b=st.floats(min_value=-1000, max_value=1000)
)
def test_division_properties(a, b):
"""Test division with random inputs ๐ฒ"""
result = divide_numbers(a, b)
if b != 0:
# โ
Should return a number
assert isinstance(result, float)
# โ
Multiplication should reverse division
assert abs((result * b) - a) < 0.0001
else:
# โ
Should handle division by zero
assert result is None
# ๐ Run the test
test_division_properties()
print("โจ All tests passed!")
๐ก Practical Examples
๐ Example 1: Testing a Shopping Cart API
Letโs fuzz test a shopping cart system:
import random
import string
from datetime import datetime
# ๐๏ธ Our shopping cart class
class ShoppingCart:
def __init__(self):
self.items = []
self.total = 0.0
def add_item(self, name, price, quantity):
"""Add item to cart ๐"""
if price < 0:
raise ValueError("Price cannot be negative! ๐ธ")
if quantity <= 0:
raise ValueError("Quantity must be positive! ๐ฆ")
if len(name) > 100:
raise ValueError("Item name too long! ๐")
self.items.append({
'name': name,
'price': price,
'quantity': quantity,
'emoji': '๐๏ธ'
})
self.total += price * quantity
print(f"โ
Added {quantity}x {name} @ ${price:.2f}")
def get_total(self):
"""Calculate total with tax ๐ฐ"""
tax_rate = 0.08 # 8% tax
return self.total * (1 + tax_rate)
# ๐ฒ Fuzz input generator
def generate_fuzz_product():
"""Generate random product data ๐ฐ"""
# Mix of valid and edge-case inputs
names = [
generate_random_string(random.randint(1, 50)), # Normal
generate_random_string(150), # Too long
"", # Empty
"๐ฎ " + generate_random_string(10), # With emoji
" " * 10, # Just spaces
]
prices = [
random.uniform(0.01, 1000), # Normal
-random.uniform(0.01, 100), # Negative
0, # Zero
float('inf'), # Infinity
random.uniform(0, 0.001), # Very small
]
quantities = [
random.randint(1, 100), # Normal
0, # Zero
-random.randint(1, 10), # Negative
random.randint(1000, 10000), # Very large
]
return {
'name': random.choice(names),
'price': random.choice(prices),
'quantity': random.choice(quantities)
}
# ๐งช Fuzz testing function
def fuzz_test_cart(iterations=100):
"""Run fuzz tests on shopping cart ๐ฏ"""
bugs_found = 0
for i in range(iterations):
cart = ShoppingCart()
product = generate_fuzz_product()
try:
cart.add_item(
product['name'],
product['price'],
product['quantity']
)
# Check invariants
assert cart.total >= 0, "Total should never be negative"
assert len(cart.items) > 0, "Items should be added"
except (ValueError, AssertionError) as e:
# Expected errors for invalid inputs
print(f"โ Caught expected error: {e}")
except Exception as e:
# Unexpected errors = bugs!
bugs_found += 1
print(f"๐ฅ BUG FOUND with input: {product}")
print(f" Error: {e}")
print(f"\n๐ฏ Fuzz testing complete!")
print(f" Iterations: {iterations}")
print(f" Bugs found: {bugs_found} ๐")
# ๐ Run the fuzzer!
fuzz_test_cart(50)
๐ฏ Try it yourself: Add more edge cases like Unicode characters or SQL injection attempts!
๐ฎ Example 2: Fuzzing a Game Score Validator
Letโs test a game scoring system:
import random
import string
# ๐ Game score validator
class GameScoreValidator:
def __init__(self):
self.valid_usernames = set()
self.scores = {}
self.achievements = {
'first_win': '๐',
'high_scorer': '๐',
'speed_demon': 'โก',
'perfectionist': '๐ฏ'
}
def register_player(self, username):
"""Register a new player ๐ฎ"""
# Validation rules
if not username:
raise ValueError("Username cannot be empty! ๐ข")
if len(username) > 20:
raise ValueError("Username too long! ๐")
if not username.replace('_', '').isalnum():
raise ValueError("Username must be alphanumeric! ๐ค")
if username.lower() in ['admin', 'root', 'system']:
raise ValueError("Reserved username! ๐ซ")
self.valid_usernames.add(username)
self.scores[username] = []
return f"โ
Player {username} registered!"
def submit_score(self, username, score, time_seconds):
"""Submit a game score ๐ฏ"""
if username not in self.valid_usernames:
raise ValueError("Player not registered! ๐ป")
if not isinstance(score, (int, float)):
raise TypeError("Score must be a number! ๐ข")
if score < 0 or score > 999999:
raise ValueError("Invalid score range! ๐")
if time_seconds <= 0:
raise ValueError("Time must be positive! โฑ๏ธ")
self.scores[username].append({
'score': score,
'time': time_seconds,
'timestamp': datetime.now()
})
# Award achievements
achievements = []
if len(self.scores[username]) == 1:
achievements.append(self.achievements['first_win'])
if score > 10000:
achievements.append(self.achievements['high_scorer'])
if time_seconds < 60:
achievements.append(self.achievements['speed_demon'])
return achievements
# ๐ฒ Fuzz input generators
def fuzz_username():
"""Generate random usernames ๐ค"""
strategies = [
lambda: ''.join(random.choices(string.ascii_letters + string.digits + '_', k=random.randint(1, 30))),
lambda: '', # Empty
lambda: 'admin', # Reserved
lambda: '๐ฎplayer', # With emoji
lambda: '../../../etc/passwd', # Path traversal attempt
lambda: "'; DROP TABLE users; --", # SQL injection
lambda: ' ' * 10, # Spaces
lambda: '\n\r\t', # Special chars
]
return random.choice(strategies)()
def fuzz_score():
"""Generate random scores ๐ฏ"""
return random.choice([
random.randint(0, 100000), # Normal
-random.randint(1, 1000), # Negative
float('inf'), # Infinity
float('nan'), # Not a number
"100", # String number
None, # Null
[1, 2, 3], # List
{"score": 100}, # Dict
])
def fuzz_time():
"""Generate random time values โฑ๏ธ"""
return random.choice([
random.uniform(1, 3600), # Normal (1s to 1h)
0, # Zero
-random.uniform(1, 100), # Negative
float('inf'), # Infinity
0.0001, # Very small
])
# ๐งช The fuzzer
def fuzz_test_game_validator(iterations=100):
"""Fuzz test the game validator ๐ฎ"""
validator = GameScoreValidator()
stats = {
'registration_errors': 0,
'score_errors': 0,
'unexpected_errors': 0,
'successful_ops': 0
}
print("๐ Starting fuzz test...\n")
for i in range(iterations):
# Try registering a player
username = fuzz_username()
try:
result = validator.register_player(username)
stats['successful_ops'] += 1
# Try submitting scores
for _ in range(random.randint(1, 5)):
score = fuzz_score()
time = fuzz_time()
try:
achievements = validator.submit_score(username, score, time)
stats['successful_ops'] += 1
if achievements:
print(f"๐ {username} earned: {' '.join(achievements)}")
except (ValueError, TypeError) as e:
stats['score_errors'] += 1
except Exception as e:
stats['unexpected_errors'] += 1
print(f"๐ฅ Unexpected error: {e}")
print(f" Input: user={username}, score={score}, time={time}")
except ValueError as e:
stats['registration_errors'] += 1
except Exception as e:
stats['unexpected_errors'] += 1
print(f"๐ฅ Unexpected registration error: {e}")
print(f" Username: {repr(username)}")
# ๐ Print statistics
print("\n๐ Fuzz Test Results:")
print(f" Total iterations: {iterations}")
print(f" โ
Successful operations: {stats['successful_ops']}")
print(f" โ Registration errors: {stats['registration_errors']}")
print(f" โ Score submission errors: {stats['score_errors']}")
print(f" ๐ฅ Unexpected errors: {stats['unexpected_errors']}")
if stats['unexpected_errors'] == 0:
print("\n๐ No unexpected errors found! Your code is robust! ๐ช")
else:
print("\nโ ๏ธ Found some bugs to fix! ๐")
# ๐ฎ Run the fuzzer!
fuzz_test_game_validator(100)
๐ Advanced Concepts
๐งโโ๏ธ Advanced Fuzzing Strategies
When youโre ready to level up, try these advanced techniques:
# ๐ฏ Smart fuzzing with mutation
class SmartFuzzer:
def __init__(self):
self.interesting_values = [
0, 1, -1, # Boundary values
'', ' ', '\n', '\t', # Whitespace
'\\', '/', '../', # Path separators
'<script>', '{{', '${', # Injection attempts
'\x00', '\xff', # Null bytes
'๐ฎ', 'ไฝ ๅฅฝ', 'ู
ุฑุญุจุง', # Unicode
]
def mutate_string(self, base_string):
"""Mutate a string in interesting ways ๐งฌ"""
strategies = [
lambda s: s[::-1], # Reverse
lambda s: s * random.randint(1, 100), # Repeat
lambda s: s.replace(random.choice(s) if s else 'a', ''), # Delete char
lambda s: s + random.choice(self.interesting_values), # Append
lambda s: ''.join(random.sample(s, len(s)) if s else ''), # Shuffle
]
mutated = base_string
for _ in range(random.randint(1, 3)):
strategy = random.choice(strategies)
try:
mutated = strategy(mutated)
except:
pass # Some mutations might fail
return mutated
def generate_edge_cases(self, data_type):
"""Generate edge cases for different types ๐ฏ"""
if data_type == 'string':
return [
'A' * 10000, # Very long
'', # Empty
'\x00' * 10, # Null bytes
''.join(chr(i) for i in range(128)), # All ASCII
'๐' * 100, # Emojis
]
elif data_type == 'number':
return [
0, -0, 1, -1,
float('inf'), float('-inf'), float('nan'),
2**31 - 1, -2**31, # 32-bit boundaries
2**63 - 1, -2**63, # 64-bit boundaries
]
elif data_type == 'list':
return [
[], # Empty
[None] * 1000, # Large with nulls
list(range(10000)), # Large sequential
[[[[[[]]]]], # Deeply nested
[1, 'a', None, True, 3.14, []], # Mixed types
]
# ๐ช Using the smart fuzzer
fuzzer = SmartFuzzer()
def test_json_parser(json_string):
"""Test a JSON parser with fuzzing ๐"""
import json
try:
result = json.loads(json_string)
return True
except json.JSONDecodeError:
return False
except Exception as e:
print(f"๐ฅ Unexpected error: {type(e).__name__}: {e}")
return None
# Generate test cases
base_json = '{"name": "test", "value": 123}'
for i in range(10):
mutated = fuzzer.mutate_string(base_json)
result = test_json_parser(mutated)
print(f"Test {i}: {result} - {repr(mutated[:50])}")
๐๏ธ Building a Reusable Fuzzing Framework
For the brave developers, hereโs a mini fuzzing framework:
import time
import traceback
from typing import Callable, Any, Dict, List
# ๐ Fuzzing framework
class FuzzTestRunner:
def __init__(self, target_function: Callable):
self.target = target_function
self.crashes = []
self.slow_inputs = []
self.interesting_inputs = []
def run(self, input_generator: Callable, iterations: int = 1000):
"""Run fuzz tests with monitoring ๐ฏ"""
print(f"๐ Starting fuzz test of {self.target.__name__}")
for i in range(iterations):
if i % 100 == 0:
print(f" Progress: {i}/{iterations} iterations... ๐")
# Generate input
test_input = input_generator()
# Run with monitoring
start_time = time.time()
try:
result = self.target(**test_input) if isinstance(test_input, dict) else self.target(test_input)
duration = time.time() - start_time
# Check for slow execution
if duration > 1.0:
self.slow_inputs.append({
'input': test_input,
'duration': duration
})
print(f"๐ Slow execution: {duration:.2f}s")
# Check for interesting results
if self._is_interesting(result):
self.interesting_inputs.append({
'input': test_input,
'result': result
})
except Exception as e:
# Caught a crash!
self.crashes.append({
'input': test_input,
'error': str(e),
'traceback': traceback.format_exc()
})
print(f"๐ฅ Crash found! {type(e).__name__}: {e}")
self._print_summary()
def _is_interesting(self, result):
"""Determine if a result is interesting ๐"""
# Customize based on your needs
if result is None:
return True
if isinstance(result, (list, dict, str)) and len(result) > 1000:
return True
return False
def _print_summary(self):
"""Print test summary ๐"""
print("\n" + "="*50)
print("๐ FUZZ TEST SUMMARY")
print("="*50)
print(f"๐ฅ Crashes found: {len(self.crashes)}")
print(f"๐ Slow inputs: {len(self.slow_inputs)}")
print(f"๐ฏ Interesting results: {len(self.interesting_inputs)}")
if self.crashes:
print("\nโ Sample crashes:")
for crash in self.crashes[:3]:
print(f" Input: {crash['input']}")
print(f" Error: {crash['error']}\n")
# ๐ฎ Example usage
def vulnerable_function(text: str, multiplier: int = 1):
"""A function with hidden bugs ๐"""
if 'admin' in text.lower():
raise PermissionError("Admin access detected!")
if multiplier > 100:
# Simulating a performance issue
time.sleep(2)
result = text * multiplier
if len(result) > 10000:
# Memory issue simulation
raise MemoryError("Result too large!")
return result
# Input generator
def generate_test_input():
return {
'text': random.choice([
'hello',
'ADMIN',
'๐ฎ' * 100,
'x' * 1000,
''
]),
'multiplier': random.choice([
1, 10, 100, 1000, -1, 0
])
}
# Run the fuzzer!
fuzzer = FuzzTestRunner(vulnerable_function)
fuzzer.run(generate_test_input, iterations=500)
โ ๏ธ Common Pitfalls and Solutions
๐ฑ Pitfall 1: Not Handling All Edge Cases
# โ Wrong way - assumes valid input
def parse_config(config_string):
lines = config_string.split('\n')
config = {}
for line in lines:
key, value = line.split('=') # ๐ฅ Crashes if no '='!
config[key] = value
return config
# โ
Correct way - handle edge cases
def parse_config(config_string):
if not config_string:
return {}
config = {}
lines = config_string.split('\n')
for line in lines:
line = line.strip()
if not line or '=' not in line:
continue # Skip invalid lines
key, _, value = line.partition('=')
key = key.strip()
value = value.strip()
if key: # Only add if key is not empty
config[key] = value
return config
# ๐งช Fuzz test it!
def fuzz_config_parser():
test_inputs = [
"", # Empty
"key=value", # Normal
"key=", # Empty value
"=value", # Empty key
"no equals sign", # Missing =
"key1=value1\n\n\nkey2=value2", # Multiple newlines
"๐=๐ฏ", # Unicode
]
for input_str in test_inputs:
try:
result = parse_config(input_str)
print(f"โ
Parsed: {repr(input_str[:20])} โ {result}")
except Exception as e:
print(f"โ Failed: {repr(input_str[:20])} โ {e}")
fuzz_config_parser()
๐คฏ Pitfall 2: Insufficient Input Diversity
# โ Poor fuzzing - only tests happy path
def weak_fuzzer():
return random.randint(1, 100) # Only positive integers!
# โ
Better fuzzing - diverse inputs
def strong_fuzzer():
return random.choice([
random.randint(-1000000, 1000000), # Wide range
0, # Zero
-1, # Negative one
2**31 - 1, # Max 32-bit int
-2**31, # Min 32-bit int
float('inf'), # Infinity
float('-inf'), # Negative infinity
float('nan'), # Not a number
])
# ๐ฏ Domain-specific fuzzing
def email_fuzzer():
"""Generate various email-like strings ๐ง"""
return random.choice([
"[email protected]", # Valid
"user@", # Missing domain
"@example.com", # Missing local part
"user@@example.com", # Double @
"user@example", # Missing TLD
"", # Empty
" " * 10, # Spaces
"[email protected]", # With tag
"[email protected]", # Subdomain
"๐ฎ@example.com", # Unicode
"[email protected]" + " " * 1000, # Long trailing spaces
"../../../etc/[email protected]", # Path traversal
"'; DROP TABLE users; [email protected]", # SQL injection
])
๐ ๏ธ Best Practices
- ๐ฏ Start Small: Begin with simple fuzzers, then add complexity
- ๐ Log Everything: Keep detailed logs of what inputs caused issues
- ๐ Automate: Run fuzz tests in CI/CD pipelines
- ๐จ Mix Strategies: Combine random generation with mutation
- โฑ๏ธ Set Timeouts: Prevent infinite loops in your tests
- ๐ Reproduce Bugs: Save failing inputs for regression tests
๐งช Hands-On Exercise
๐ฏ Challenge: Build a Password Strength Checker Fuzzer
Create a comprehensive fuzz tester for a password strength checker:
๐ Requirements:
- โ Test with various character types (letters, numbers, symbols, unicode)
- ๐ท๏ธ Check edge cases (empty, very long, special characters)
- ๐ค Verify consistent scoring
- ๐ Find inputs that cause crashes or unexpected behavior
- ๐จ Generate a report of interesting findings
๐ Bonus Points:
- Implement mutation-based fuzzing
- Add performance monitoring
- Create a corpus of โinterestingโ passwords
๐ก Solution
๐ Click to see solution
import random
import string
import time
from collections import defaultdict
# ๐ Password strength checker to test
class PasswordChecker:
def __init__(self):
self.common_passwords = {'password', '123456', 'qwerty', 'admin'}
def check_strength(self, password):
"""Check password strength ๐ก๏ธ"""
if not password:
return {'score': 0, 'level': 'invalid', 'emoji': 'โ'}
score = 0
feedback = []
# Length check
if len(password) >= 8:
score += 20
if len(password) >= 12:
score += 20
if len(password) >= 16:
score += 10
# Character diversity
has_lower = any(c.islower() for c in password)
has_upper = any(c.isupper() for c in password)
has_digit = any(c.isdigit() for c in password)
has_special = any(c in string.punctuation for c in password)
diversity = sum([has_lower, has_upper, has_digit, has_special])
score += diversity * 15
# Common password check
if password.lower() in self.common_passwords:
score = min(score, 10)
feedback.append("Common password detected! ๐ซ")
# Patterns check
if any(str(i)*3 in password for i in range(10)):
score -= 10
feedback.append("Repeated patterns found! ๐")
# Determine level
if score >= 80:
level = 'strong'
emoji = '๐ช'
elif score >= 60:
level = 'good'
emoji = '๐'
elif score >= 40:
level = 'fair'
emoji = '๐'
else:
level = 'weak'
emoji = '๐'
return {
'score': max(0, min(100, score)),
'level': level,
'emoji': emoji,
'feedback': feedback
}
# ๐ฒ Advanced password fuzzer
class PasswordFuzzer:
def __init__(self):
self.interesting_chars = [
'\x00', # Null byte
'\n', '\r', '\t', # Whitespace
'\\', '/', '|', # Path separators
'<', '>', '"', "'", # HTML/SQL chars
'${', '{{', '<%', # Template injection
'\u0000', '\uffff', # Unicode boundaries
'๐', '๐ฏ', '๐ฅ', # Emojis
'admin', 'root', # Common words
]
self.mutations = []
def generate_password(self):
"""Generate a fuzzed password ๐ฒ"""
strategies = [
self._random_ascii,
self._edge_cases,
self._unicode_chaos,
self._pattern_based,
self._mutation_based,
self._length_extreme,
]
return random.choice(strategies)()
def _random_ascii(self):
"""Standard random password"""
length = random.randint(1, 20)
chars = string.ascii_letters + string.digits + string.punctuation
return ''.join(random.choice(chars) for _ in range(length))
def _edge_cases(self):
"""Edge case passwords"""
return random.choice([
'', # Empty
' ', # Single space
' ' * 100, # Many spaces
'\x00' * 10, # Null bytes
'a' * 10000, # Very long
'123456', # Common
'password', # Common
'P@ssw0rd!', # "Secure" pattern
])
def _unicode_chaos(self):
"""Unicode and emoji passwords"""
length = random.randint(1, 20)
chars = []
for _ in range(length):
choice = random.randint(0, 3)
if choice == 0:
chars.append(chr(random.randint(0x0100, 0x017F))) # Latin Extended
elif choice == 1:
chars.append(chr(random.randint(0x4E00, 0x9FFF))) # CJK
elif choice == 2:
chars.append(random.choice(['๐', '๐', '๐ช', '๐ฏ', '๐']))
else:
chars.append(random.choice(string.ascii_letters))
return ''.join(chars)
def _pattern_based(self):
"""Passwords with patterns"""
patterns = [
lambda: 'abc' * random.randint(1, 10),
lambda: '123' * random.randint(1, 10),
lambda: 'a1b2c3' * random.randint(1, 5),
lambda: 'Pass' + str(random.randint(1000, 9999)),
lambda: random.choice(['!', '@', '#']) * random.randint(8, 15),
]
return random.choice(patterns)()
def _mutation_based(self):
"""Mutate a base password"""
if not self.mutations:
self.mutations.append('password123')
base = random.choice(self.mutations)
mutations = [
lambda s: s.upper(),
lambda s: s.lower(),
lambda s: s[::-1], # Reverse
lambda s: s.replace('a', '@'),
lambda s: s.replace('e', '3'),
lambda s: s.replace('o', '0'),
lambda s: s + random.choice(self.interesting_chars),
lambda s: s[:-1] if s else s, # Remove last char
]
mutated = random.choice(mutations)(base)
if len(mutated) < 100: # Don't let it grow too big
self.mutations.append(mutated)
return mutated
def _length_extreme(self):
"""Extreme length passwords"""
if random.random() < 0.5:
# Very short
return ''.join(random.choices(string.ascii_letters, k=random.randint(0, 3)))
else:
# Very long
return ''.join(random.choices(string.ascii_letters, k=random.randint(100, 1000)))
# ๐งช Comprehensive fuzz tester
def fuzz_test_password_checker(iterations=1000):
"""Run comprehensive fuzz tests ๐ฏ"""
checker = PasswordChecker()
fuzzer = PasswordFuzzer()
stats = defaultdict(int)
crashes = []
inconsistencies = []
slow_checks = []
interesting_cases = []
print("๐ Starting password checker fuzz test...\n")
for i in range(iterations):
if i % 100 == 0 and i > 0:
print(f"Progress: {i}/{iterations} ๐")
password = fuzzer.generate_password()
# Test 1: Check for crashes
start_time = time.time()
try:
result1 = checker.check_strength(password)
duration = time.time() - start_time
stats[result1['level']] += 1
# Test 2: Check for consistency
result2 = checker.check_strength(password)
if result1 != result2:
inconsistencies.append({
'password': password,
'result1': result1,
'result2': result2
})
# Test 3: Check for performance
if duration > 0.1:
slow_checks.append({
'password': password[:50] + '...' if len(password) > 50 else password,
'duration': duration
})
# Test 4: Find interesting cases
if result1['score'] == 100:
interesting_cases.append(f"Perfect score: {password[:30]}")
elif result1['score'] == 0 and len(password) > 0:
interesting_cases.append(f"Zero score (non-empty): {password[:30]}")
elif len(password) > 50 and result1['level'] == 'weak':
interesting_cases.append(f"Long but weak: {password[:30]}...")
except Exception as e:
crashes.append({
'password': password,
'error': str(e),
'type': type(e).__name__
})
# ๐ Print results
print("\n" + "="*60)
print("๐ FUZZ TEST RESULTS")
print("="*60)
print(f"\n๐ Score Distribution:")
for level, count in sorted(stats.items()):
percentage = (count / iterations) * 100
print(f" {level}: {count} ({percentage:.1f}%)")
print(f"\n๐ฅ Crashes: {len(crashes)}")
if crashes:
for crash in crashes[:3]:
print(f" Password: {repr(crash['password'][:30])}")
print(f" Error: {crash['type']}: {crash['error']}\n")
print(f"\n๐ Inconsistencies: {len(inconsistencies)}")
if inconsistencies:
for inc in inconsistencies[:3]:
print(f" Password: {repr(inc['password'][:30])}")
print(f" Results differ: {inc['result1']['score']} vs {inc['result2']['score']}\n")
print(f"\n๐ Slow checks: {len(slow_checks)}")
if slow_checks:
for slow in slow_checks[:3]:
print(f" Password: {repr(slow['password'])}")
print(f" Duration: {slow['duration']:.3f}s\n")
print(f"\n๐ฏ Interesting cases: {len(interesting_cases)}")
for case in interesting_cases[:5]:
print(f" {case}")
if not crashes and not inconsistencies:
print("\nโ
Password checker passed all fuzz tests! ๐")
else:
print("\nโ ๏ธ Issues found - review and fix! ๐ง")
# ๐ฎ Run the comprehensive test!
fuzz_test_password_checker(500)
๐ Key Takeaways
Youโve learned so much about fuzz testing! Hereโs what you can now do:
- โ Generate random test inputs with confidence ๐ช
- โ Find hidden bugs that manual testing misses ๐
- โ Build robust fuzzers for any Python code ๐ฏ
- โ Use hypothesis for property-based testing ๐งช
- โ Create custom fuzzing frameworks for your needs ๐
Remember: Fuzz testing is like having a tireless QA team that never sleeps! ๐ค
๐ค Next Steps
Congratulations! ๐ Youโve mastered fuzz testing with random input generation!
Hereโs what to do next:
- ๐ป Practice with the password checker exercise
- ๐๏ธ Add fuzz testing to your existing projects
- ๐ Explore the
hypothesis
library in depth - ๐ Share your interesting fuzzing discoveries!
Next up in our testing series: Mutation Testing: Testing Your Tests - where weโll learn how to verify that our tests are actually testing what we think they are!
Remember: Every bug found by fuzzing is one less bug your users will encounter. Keep fuzzing, keep improving, and most importantly, have fun breaking things (safely)! ๐
Happy fuzzing! ๐๐ฒโจ