Connecting to a model is the quick part. Here is the rest of the work a first version usually involves, so you can judge a proposal.
Start with one narrow job
Start with one job, such as answering shipping questions from your policy pages. A narrow first job is easier to test, cheaper to run, and shows quickly whether users want it. Also ask whether AI is needed at all. Good search, or a better form, sometimes solves the problem more reliably.
Answers from your own content
A model does not know your products or policies. A common approach, often called retrieval, finds the relevant passages from your content and passes them to the model with the question. The work includes:
- Getting the content into a clean, current form, and keeping it up to date.
- Finding the right passages. This is where most quality problems come from.
- Telling the model to say it does not know, and testing that it does, since this instruction often fails.
- Showing users where the answer came from, where that helps.
Keep the model behind your server
Keys in an app or a web page can be copied by anyone. Calls should go through your own server, which also lets you check who is asking, limit how much each person can use, and switch providers later without releasing a new app. The app itself usually changes little: often one new server endpoint and one screen.
Running costs
AI features cost money each time they run, unlike most software. Providers charge by the amount of text or images processed, and prices change, so check the current rates.
- Estimate from expected use, and then assume a popular feature gets used more.
- Limit the length of answers and the amount of context.
- Cache common answers, but only ones that contain nothing about a particular user.
- Use a cheaper model for simple steps.
- Set per-user limits and a spending alert with your provider.
Privacy
Decide this before you build:
- What is sent to the provider? This includes anything users type, which may include names, addresses or health details even if you did not ask for them.
- Does your plan allow the provider to use the data for training? Terms vary by provider and plan, so check the current ones.
- What do you store, and for how long?
- What do users see? Tell them that an AI service is involved, and what it receives. On iPhone this is a store rule: Apple’s guidelines require apps to disclose when personal data goes to third-party AI and to get explicit permission first.
Removing personal details automatically helps, but no filter catches everything, so do not promise that it does. Some sectors have their own rules, so take advice if you handle health or financial data.
When it is wrong
It will sometimes be wrong, confidently. Plan for it:
- Do not let it make promises on your behalf: prices, refunds, medical or legal advice.
- For anything consequential, have a person check the result before it takes effect.
- Give users an easy route to a person.
- Plan what happens when the provider is slow or down. A slow answer is often the first complaint, so show the answer as it arrives where you can.
- Treat what users type as untrusted. People will try to talk the model into ignoring its instructions, so it must not have access to anything it should not reveal. Documents it reads can carry hidden instructions too. Limit search to what this user may see, and check any action it takes on your server.
Testing it
Collect real questions: common ones, awkward ones, and ones it should refuse. Keep them as a test set and re-run them whenever you change the prompt, the content or the model. Fix the model version where you can, because providers update and retire models, and re-test when that happens. After launch, review a sample of real conversations, with suitable privacy controls, and keep a record of what fails.
What a first version looks like
A sensible first release is usually one endpoint and one screen, with limits, logging and a test set. If it answers from your own content, add the step that prepares that content and keeps its index current. Then look at what people actually do with it before you build more.
Our own free AI Store Assistant plugin for WooCommerce has a mode that makes no external AI calls at all, and AI modes that use the merchant’s own key. Our page for it says what may be sent. If you would like to add something similar to your product, see AI features for apps and web products.