SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillssynthetic-data-generator

synthetic-data-generator

Data & analytics

View repository

Updated 10-15 23:58

About this skill

Synthetic Data Generator (Digital Twin Simulator)

Overview

This skill gives the agent the ability to act as a Respondent Simulator (Digital Twin Simulator) in research. The agent will adopt a persona (digital twin) based on specified profile parameters and respond to research instruments (questionnaires, Likert scales, open-ended interview questions) like an actual human respondent.

In addition to generating data offline to CSV/JSON files, this skill also supports Active Participant Simulator, namely the ability to automate the filling out of online real questionnaires (such as Google Forms, Typeform, etc.) through a browser using MCP server integration chrome-devtools.

Dependencies

  • Standard Python modules (for large-scale offline data generation).
  • MCP Server chrome-devtools (for online questionnaire filling via browser).
  • Zero-Failure Auto-Provisioning Rule: If chrome-devtools is not yet active/installed when filling out the online questionnaire, the agent is FORBIDDEN from answering "not installed". The agent MUST automatically execute direct triggering on-demand: npx -y chrome-devtools-mcp@latest or py bin/mcp_auto_updater.py.

Quick Start

Example usage:

  1. Offline: "Use the synthetic-data-generator skill to create a synthetic survey dataset containing 100 respondents with the demographic profile of final-year students and responses regarding satisfaction with online mentoring services."
  2. Online: "Use the synthetic-data-generator skill to simulate 5 digital twins filling out a Google Form at the URL https://forms.gle/xxxxx with the profile of digital MSME actors."

Workflow

1. Instrument and Profile Initialization

  • Instrument: Understand the structure of the research instrument used. Is it quantitative (scale 1-5) or qualitative (narrative answers)?
  • Demographic/Psychographic Profile: Understand the distribution of the target sample. Example: "Create 300 data points with a distribution of 60% male, 40% female, age 18-35 years, high-medium level of technology skepticism."

2. Persona Prompting Strategy (LLM Adoption)

To ensure the data is not uniform and has realistic variance:

  • Use the System Prompt technique dynamically that injects random traits (traits) into each respondent iteration.
  • Apply a sufficient Temperature (for example 0.7 - 0.9) so that answers are not deterministic.
  • Logical Consistency: Ensure that answers across question items (especially those that are correlated with each other or reversed item) are consistent with the persona currently being adopted.
  • Attention Checks: If requested, insert a "trap" or natural anomaly answer (e.g., a respondent answering carelessly) with a small probability (e.g., 5%) to simulate noise in the real world.

3. Batch Execution via Python (Mandatory for Large-Scale Offline)

Calling the LLM one by one for hundreds of respondents is very inefficient in terms of tokens and time. If N > 10 and run offline, MANDATORY create a Python script that utilizes Batch Prompting.

Python Script Flow Guide:

  1. Create a centralized instruction prompt: "Generate 50 lines of CSV-formatted data. Each line represents 1 unique respondent. The columns consist of: Demographics, Q1, Q2, Q3 (scale 1-5) and Opinion (text/qualitative). Ensure diversity according to the distribution: [TARGET_PROFILE]. Output ONLY in valid CSV format without markdown."
  2. Use a loop in Python to call the LLM (for example via google-genai or the relevant LLM endpoint) N/50 times.
  3. Parse the CSV string resulting from the generate into a data structure (such as pandas.DataFrame).
  4. Combine all batches and clean rows that are error/malformed.
  5. Save the final DataFrame to synthetic_dataset.csv.

4. Online Questionnaire Filling (Active Participant Simulator)

If the user asks to fill out an online questionnaire such as Google Forms:

  1. Page Analysis (Form Mapping): Connect to the chrome-devtools server and direct the browser to the questionnaire URL using puppeteer_navigate. Identify the question types (text input, radio button, checkbox) using puppeteer_evaluate.
  2. Respondent Batching: For each digital twin (e.g., N respondents):
    • Generate the respondent's answer profile internally (in JSON/Key-Value format).
    • Find the input elements that match the questions.
    • Human Input Simulation:
      • For multiple choice/Likert scale, use puppeteer_click on the corresponding radio button.
      • For text fields, use puppeteer_type to type the answer naturally.
      • Add a random time delay (human-like delay) between 1.5 and 3 seconds before clicking the submit/next button to mimic human behavior and avoid anti-bot detection.
    • Click the "Send" / "Submit" button using puppeteer_click.
    • Take a screenshot of the successful submission page (puppeteer_screenshot) as validation evidence.
    • If you want to fill in the next respondent, click the "Submit another response" link to load a new form.

5. Final Validation and Handover

  • Offline: Perform a simple describe() via script to ensure the Likert values are within the valid range (example: min 1, max 5) and there are no missing values that are odd.
  • Online: Provide a summary log of successful form submissions (e.g., "3/3 Respondents successfully filled out the Google Form") along with screenshot files as proof of submission for each respondent.

Common Mistakes & Critical Rules

  • Data Too Uniform (No Variance): Producing monotone or variation-less data responses because the LLM temperature (temperature) is set to 0 or the instruction system is too rigid.
  • Persona Inconsistency: Ignoring the internal logic of respondents (for example, a respondent with a "tech-illiterate/tech-skeptical" profile yet giving responses that are highly skilled and enthusiastic about AI adoption).
  • Malformed File Format (Malformed CSV): Saving output in an invalid CSV format so that it fails to be read by the data processing agent.
  • Bot Detection on Online Forms: Filling out forms too quickly without time intervals, which triggers the appearance of CAPTCHA. Always use randomized delay (randomized delay) when mimicking human input.