What is Midscene.js
Midscene.js is an automation SDK based on AI technology that simplifies commands in UI automation testing by using large language models (LLM). Users describe interaction steps or expected data formats in natural language, and Midscene.js will execute the corresponding operations. Midscene.js supports performing actions, querying data, and page assertions, reducing script maintenance costs and providing visual reports. Midscene.js is an open-source project that can run in the user's own environment, ensuring data security.
Main features of Midscene.js
- Natural language interaction: Users execute a series of actions based on described steps, such as entering text, clicking buttons, etc.
- Custom data query: Users describe the required JSON format, and Midscene.js will extract the corresponding data based on the page content.
- Intuitive inference: Users make assertions about the page state to ensure the UI meets expectations.
- Chrome extension: Supports users in using natural language to execute actions, queries, and assertions on any web page without writing code.
- Code project integration: Integrates with tools such as YAML scripts, Puppeteer, and Playwright to achieve automated testing.
- Visual reports: Provides a visual report after each run, including animation playback and step details, as well as a testing environment for users to adjust prompts without rerunning the script.
Technical principles of Midscene.js
- Natural language processing (NLP): Receives the user's natural language input and parses instructions based on NLP technology.
- Interface understanding: The model analyzes web page content and understands the interface elements and actions described by the user.
- Execute actions: Based on the understanding results, the model will execute the corresponding browser automation operations, such as simulating user input, clicks, etc.
- Data extraction: The model extracts data in a specific format from the web page based on the user's query request.
- Assertion verification: The model checks whether the page state meets the user's assertion conditions.
Midscene.js project address
- Project official website：midscenejs.com
- GitHub repository：https://github.com/web-infra-dev/midscene
Application scenarios of Midscene.js
- Automated testing scenarios: Automatically execute test cases to ensure that each function of the website or application works as expected.
- Data scraping scenarios: Extract structured data from web pages, such as product information, for use in market analysis or inventory management.
- Performance monitoring scenarios: Simulate user behavior, monitor website performance, and promptly discover and resolve performance bottlenecks.
- Interface consistency check scenarios: Verify whether the website displays consistently on different devices and browsers to ensure user experience.
- Price monitoring scenarios: Regularly check product prices and automatically notify users or the system when price changes are detected.