1 Contents 1. AI module in a full product system 2. How to start when building a full product system 3. Integration Pipeline 4. Automated testing 5. Human Intervention in AI Systems 6. Ensure Prototype Technology Scales 7. Cloud deployment 8. Continuing the Feedback Loop
2 1. AI module in a full product system • are all modules in the Facebook system intelligent? • Login • Accept a friend request • Post new story • Post new status • Tag friends in a picture
3 Customer journey: share an image
4 User story • Step 1: User click photo button to choose images to share • Step 2: Select any images in this device • Step 3: • Auto suggest faces of people in friend list appeared in images • When the user finds a friend name in the tag button, display the nickname of the person in priority of related activities
🡪 Correct spelling not already to do so integrate it another time
5 2. How to start when building a full product system
• Is the prototype good enough?
• Is the prototype meet the requirement? • Actions AI team • Find the way to develop AI core to meet the requirement of the product • Build API to integrate with the full product system
6 7 3. Integration Pipeline
Develop Test Staging Product
environment environment environment environment
8 9 • The development environment: enables internal developers and product managers to access the latest version of the code • primarily used to ensure feature correctness
Development environment
10 Test environment Test environment: which is configured similarly to the production environment and runs new code and models that will eventually become production if they pass all quality assurance checks Typically, a test environment meets the minimum requirements for your application to function. - Meet new requirements? - UI? - Workflow of User? - result?
11 Example: Login Page Type- Negative/ Positive Sr. No. Functional Test Cases Test Case 1 Verify if a user will be able to login with a valid username and valid password. Positive
2 Verify if a user cannot login with a valid username and an invalid password. Negative
3 Verify the login page for both, when the field is blank and Submit button is clicked. Negative
4 Verify the ‘Forgot Password’ functionality. Positive
5 Verify the messages for invalid login. Positive 6 Verify the ‘Remember Me’ functionality. Positive 7 Verify if the data in password field is either visible as asterisk or bullet signs. Positive
8 Verify if a user is able to login with a new password only after he/she has changed the Positive password. 9 Verify if the login page allows to log in simultaneously with different credentials in a different Positive browser. 10 Verify if the ‘Enter’ key of the keyboard is working correctly on the login page. Positive Other Test Cases 11 Verify the time taken to log in with a valid username and password. Performance & Positive Testing 12 Verify if the font, text color, and color coding of the Login page is as per the standard. UI Testing & Positive Testing 13 Verify if there is a ‘Cancel’ button available to erase the entered text. Usability Testing 14 Verify the login page and all its controls in different browsers Browser Compatibility & Positive Testing. 12 Staging environment • Staging is when you create an instance of an application that you're confident enough to show the immediate owner (sponsor) but not users. Before exposing to the latter group, you'd want to run more tests. A staging environment is meant to simulate production as much as possible. • The staging environment is often restricted to a few select people. Whitelisted emails, IPs, and your developer team are the only groups that can use the application in staging.
13 Product environment • Production environment: the live environment that end users are actively using
14 4. Automated testing • Code must be validated each time it is pushed in order for us to have trust in our continuous integration
• To test this module, we need to test 3 cases:
• amount > 1oo • amount <100 • amount = 100 🡪 Prepare test case
15 Test types • Unit tests: validate self-contained pieces of code. • Integration test: validate code that controls the interactions between systems—for instance, code that talks to a database or a third-party API. • Acceptance tests: validate the system as a whole, testing functionality in ways that an end user would use the system. These can be direct methods like user interface inputs or programmatic API calls if your solution is an API.
16 5. Human Intervention in AI Systems
Sample Chatbot Architecture that Includes a Human in the Loop
17 6. Ensure Prototype Technology Scales As the number of users or transactions per second grows, technology that has poor scalability becomes a deal-breaker.
• Performance test • Load test
18 Performance test • Performance Testing is a software testing process used for testing the speed, response time, stability, reliability, scalability, and resource usage of a software application under a particular workload. The main purpose of performance testing is to identify and eliminate the performance bottlenecks in the software application. It is a subset of performance engineering and is also known as “Perf Testing”. • Speed – Determines whether the application responds quickly • Scalability – Determines maximum user load the software application can handle. • Stability – Determines if the application is stable under varying loads
19 Load test - Scalability
Load test: to determine how many
users your system can actually handle. You can configure tests to simulate various user scenarios which can focus on different parts of your system (such as a checkout page, for example). You can determine how the load behaves when coming from different geo- locations or how the load might build up, then level out to a sustained level.
23 AWS service • Computing • Customer do not build their own application server • Use on demand • Cost depends on Ram, CPU, GPU, time to use?
• Storage • Customer do not build their own database server • Use on demand • Cost depends on Ram, CPU, GPU, number of requests, space to use? • Always consider service level agreements (SLAs).
24 8. Continuing the Feedback Loop • How is a successful AI module • Chatbot? • Recommendation System? • Person recognition?
🡪 stored information to know how AI module supports users
25 Chatbot
26 Pitfalls • End Users Resist Adopting the Technology
• Micromanaging the Development Team
• Not Having the Correct Skills Available
27 Action checklist
___ Reevaluate user stories to ensure that they are still relevant. ___ Establish a continuous integration pipeline with automated tests to ensure system quality. ___ Allow the system to involve human intervention as necessary. ___ Perform load testing on your system to ensure that it and its components are scalable. ___ If your system is deployed in the cloud, review the SLAs and make sure they are sufficient for your user stories. ___ Release the live production system to users and begin the feedback lifecycle process.
