How to Use TensorFlow AI for Negative Testing with Selenium?

Negative testing is a critical process in software testing where the system is validated against invalid, unexpected, or malicious inputs. It ensures applications behave as intended, even under adverse conditions, thereby enhancing robustness and user satisfaction.

However, manual negative testing can be time-consuming and error-prone. With the advent of Artificial Intelligence (AI), testers can now automate and optimize this process, achieving better accuracy and efficiency. This article explores the importance of negative testing, what it achieves, and how AI can revolutionize this practice.

---

Why Negative Testing is Necessary

Negative testing validates a system’s resilience by exposing it to incorrect inputs, such as invalid email addresses, unsupported options, or missing fields. It ensures the application:

• Handles errors gracefully.
• Provides meaningful feedback to users.
• Prevents crashes or vulnerabilities.

For example:

• Scenario: A login form expects a valid email and password. Negative testing involves inputting an invalid email to verify the application displays the appropriate error message.

Code Example: A Negative Test Case

from selenium import webdriver
from selenium.webdriver.common.by import By

# Test invalid input for a login form
driver = webdriver.Chrome()
driver.get("https://example.com/login")

# Input invalid email and password
driver.find_element(By.ID, "email").send_keys("invalid-email")
driver.find_element(By.ID, "password").send_keys("123")
driver.find_element(By.ID, "submit").click()

# Check for error message
error_message = driver.find_element(By.CLASS_NAME, "error").text
print(f"Error Message: {error_message}")
driver.quit()

Expected Output:

Error Message: Invalid email format

For a detailed guide on performing negative testing, refer to How to Perform Negative Testing with Selenium and Python.

---

What We Achieve Through Negative Testing

Negative testing helps achieve:

1. System Stability: Validates the application remains functional under invalid inputs.
2. Improved User Experience: Ensures meaningful error messages guide users.
3. Security Assurance: Identifies vulnerabilities, such as injection attacks.
4. Robustness: Verifies the system can handle edge cases and unexpected scenarios.

---

How AI Helps Validate Data in Negative Testing

AI automates and enhances negative testing by:

1. Automating Input Classification:
   • AI can predict whether inputs are valid or invalid using trained models.
2. Enhancing Test Case Generation:
   • AI dynamically generates diverse test cases, including edge cases.
3. Predicting Failures:
   • By analyzing patterns, AI can predict which inputs are likely to cause errors.
4. Real-Time Feedback:
   • AI validates inputs during execution, reducing manual effort.

Code Example: AI Integration for Input Validation

import tensorflow as tf
import numpy as np

# Load the pre-trained AI model
model = tf.keras.models.load_model("form_validation_model.h5")

# Example input: [Name Valid, Email Valid, Website Valid, Comment Valid, Gender Valid, Country Valid]
input_data = np.array([[1, 0, 1, 1, 0, 1]])  # Invalid email and gender

# Predict the outcome
prediction = model.predict(input_data)
result = "success" if prediction[0][0] >= 0.5 else "error"
print(f"AI Prediction: {result}")

Expected Output:

AI Prediction: error

Explanation: The AI model evaluates the input and classifies it based on training data. In this case, the invalid email and gender values result in a prediction of “error.”

Why and How to Integrate AI into Testing Workflows

Why Use AI for Testing?

• AI reduces manual effort by automating repetitive tasks.
• It identifies edge cases and patterns missed by manual testing.
• It improves test accuracy and scalability.

How to Integrate AI:

1. Collect a Dataset:
   • Gather examples of valid and invalid inputs.
2. Train an AI Model:
   • Use machine learning frameworks like TensorFlow to train a model to classify inputs.
3. Integrate the Model:
   • Embed the AI model into your testing framework to validate inputs dynamically.
4. Continuously Improve:
   • Regularly update the dataset and retrain the model to handle new scenarios.

Code Example: AI-Powered Workflow

def validate_form_with_ai(data):
    input_data = np.array([[data["name_valid"], data["email_valid"], data["website_valid"],
                            data["comment_valid"], data["gender_valid"], data["country_valid"]]])
    prediction = model.predict(input_data)
    return "success" if prediction[0][0] >= 0.5 else "error"

# Example usage
data = {
    "name_valid": 1,
    "email_valid": 0,  # Invalid email
    "website_valid": 1,
    "comment_valid": 1,
    "gender_valid": 0,  # Invalid gender
    "country_valid": 1
}
print(validate_form_with_ai(data))  # Expected Output: error

Challenges and Best Practices

Challenges:

• Requires a well-labeled dataset for training.
• Initial setup can be complex.

Best Practices:

• Start with simple datasets and models.
• Combine AI predictions with manual validations for critical cases.
• Continuously retrain models to adapt to new input patterns.

---

Complete Code for AI-Powered Negative Testing

Here’s the complete code implementation of an AI-powered negative testing framework. This code integrates Selenium for automated UI testing and a TensorFlow model to validate input data dynamically.

import tensorflow as tf
import numpy as np
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException

# Load the AI model for input validation
model = tf.keras.models.load_model("form_validation_model.h5")

# Function to validate inputs using AI
def validate_with_ai(data):
    input_data = np.array([[data["name_valid"], data["email_valid"], data["website_valid"],
                            data["comment_valid"], data["gender_valid"], data["country_valid"]]])
    prediction = model.predict(input_data)
    return "success" if prediction[0][0] >= 0.5 else "error"

# Function to safely interact with web elements
def safe_find_element(driver, by, value, description):
    try:
        return driver.find_element(by, value)
    except NoSuchElementException:
        print(f"Element not found: {description} ({value})")
        return None

# Function to perform negative testing
def perform_negative_test_case(data, expected_result):
    driver = webdriver.Chrome()
    driver.get("https://example.com/form")  # Replace with the actual URL of the form

    try:
        # Fill form fields
        if "name" in data:
            name_field = safe_find_element(driver, By.NAME, "name", "Name field")
            if name_field:
                name_field.clear()
                name_field.send_keys(data["name"])

        if "email" in data:
            email_field = safe_find_element(driver, By.NAME, "email", "Email field")
            if email_field:
                email_field.clear()
                email_field.send_keys(data["email"])

        if "website" in data:
            website_field = safe_find_element(driver, By.NAME, "website", "Website field")
            if website_field:
                website_field.clear()
                website_field.send_keys(data["website"])

        if "comment" in data:
            comment_field = safe_find_element(driver, By.NAME, "comment", "Comment field")
            if comment_field:
                comment_field.clear()
                comment_field.send_keys(data["comment"])

        if "gender" in data:
            gender_radio = safe_find_element(driver, By.XPATH, f"//input[@name='gender'][@value='{data['gender']}']", "Gender field")
            if gender_radio:
                gender_radio.click()

        if "country" in data:
            country_dropdown = safe_find_element(driver, By.NAME, "country", "Country dropdown")
            if country_dropdown:
                select = Select(country_dropdown)
                select.select_by_visible_text(data["country"])

        # Submit the form
        submit_button = safe_find_element(driver, By.NAME, "submit", "Submit button")
        if submit_button:
            submit_button.click()

        # Validate AI result
        ai_result = validate_with_ai({
            "name_valid": 1 if data.get("name") else 0,
            "email_valid": 1 if "@" in data.get("email", "") else 0,
            "website_valid": 1 if data.get("website", "").startswith("http") else 0,
            "comment_valid": 1 if data.get("comment") else 0,
            "gender_valid": 1 if data.get("gender") in ["male", "female", "Other"] else 0,
            "country_valid": 1 if data.get("country") in ["USA", "Canada", "UK"] else 0
        })

        print(f"AI Result: {ai_result}")

        # Final assertion
        assert ai_result == expected_result, f"Test Failed: Expected {expected_result}, got {ai_result}"

    finally:
        driver.quit()

# Example test case
test_data = {
    "name": "",
    "email": "invalid-email@",  # Invalid email
    "website": "http://invalid-url",  # Invalid URL
    "comment": "",
    "gender": "InvalidGender",  # Unsupported gender
    "country": "InvalidCountry"  # Unsupported country
}

# Perform the test
perform_negative_test_case(test_data, "error")

Result:

Note: for trained model dataset, and other codes, Here is the git repo URL:

https://github.com/itlearn360/automationtestingwithaitensorflow/tree/main/negative_testing

Conclusion

Negative testing is vital for building robust, secure, and user-friendly applications. By integrating AI into the testing process, teams can enhance efficiency, cover edge cases, and improve accuracy. Start your journey into AI-powered testing today, and for more insights on traditional methods, check out this blog.