LinkT&B Editorial Department
Importing documents into an AI system does not establish that it is ready for business use. Evaluate evidence, document permissions, version freshness and review time to identify operational problems during a limited trial.
Contents
- Failure cases to include in a pilot
- Pitfall 1: Underestimating data preparation
- Pitfall 2: Lack of ROI design
- Pitfall 3: Defer security and governance
- Pitfall 4: Field resistance and change management
- Pitfall 5: Undesigned operational system
- A bird’s-eye view of the five pitfalls: Common root causes
- Why small and medium-sized businesses are advantageous
- Summary: List of things to do before PoC
- Please contact us
- Reference materials
- Related Articles
- Discuss your project
“It's about time we introduced generative AI.” From 2025 to 2026, fewer companies will not hear these words from management. Since the introduction of ChatGPT, not only large companies but also small and medium-sized enterprises are rapidly considering the introduction of generative AI. However, according to the Ministry of Internal Affairs and Communications' “Information and Communications White Paper 2020 Edition,” 42.7% of Japanese companies have already formulated a policy for utilizing generative AI, and 46.8% are currently using it in specific operations, but the number of companies that are actually producing results is still limited. BCG's 2024 survey reports that approximately 74% of companies are struggling to create value and scale AI in general, and the reality is that the path from PoC to production is more difficult than expected.
In this article, we will summarize the "5 pitfalls" that many companies fall into when introducing generative AI in-house, and explain how to avoid each at a practical level. In particular, we place emphasis on how small and medium-sized enterprises with few dedicated IT departments can achieve results with limited budgets and personnel.
Figure 1. Correlation of five pitfalls in implementing generative AI. If you overlook even one of them, you are likely to end up with a PoC.
Failure cases to include in a pilot
| Item | What to check | How to assess it |
|---|---|---|
| Missing evidence | A question absent from the documents | Can the system abstain? |
| Permissions | Questions about restricted documents | Do retrieval and summaries respect access? |
| Updates | Old and new procedures coexist | Does the answer identify the applicable version? |
Pitfall 1: Underestimating data preparation
The output quality of generative AI depends not only on the ability of the model but also on the quality of the input data. Despite this, many PoC projects spend time selecting models while neglecting the organization and cleansing of internal data.
Typical failure patterns are: This is a case where in-house manuals and meeting minutes are directly input into RAG (Retrieval-Augmented Generation), but old regulations and obsolete procedure manuals are mixed in, resulting in a significant drop in the accuracy of AI responses. Alternatively, there are cases where a large number of erroneous conversions occur when a PDF paper form is read using OCR, and the cost of manually correcting it ends up exceeding the PoC budget.
Workaround
First, narrow down your target business to one and organize data only for the documents used in that business. If you try to transform the entire company's knowledge into AI at once, it is not uncommon for it to take several months just to prepare the data, depending on the scope of the data and the state of documentation. The golden rule is to start with tasks that have a limited scope of data, such as responding to internal FAQs or drafting standard reports.
Pitfall 2: Lack of ROI design
If you secure a budget with only a vague expectation that “Introducing AI will increase productivity,” you won't be able to answer the question “So, how much profit did you make?” after the PoC ends, and the approval process for going live will not be passed.
To measure the ROI of generative AI, it is necessary to quantitatively record the baseline before implementation. For example, if it used to take an average of 15 minutes to respond to each inquiry, but it can be reduced to 8 minutes with AI support, this can be converted into a monetary amount by multiplying the number of inquiries handled per month by 7 minutes. However, if this baseline measurement is not performed before PoC, there is no way to prove the effectiveness.
Workaround
Before starting the PoC, please define "what, in what units, and by how much improvement should be considered a success." KPIs are limited to indicators that can be measured in existing operations, such as processing time, error rate, and number of cases handled. The reality is that qualitative indicators such as “AI has increased creativity” are difficult to use as decision-making material for management.
Pitfall 3: Defer security and governance
When using cloud-based generative AI services, it is important to note that data handling differs depending on the type of service. Input data may be used for learning in personal services (such as ChatGPT's free plan), but in enterprise/API plans (such as Azure OpenAI and Google Cloud's Gemini API), it is clearly stated that the input data will not be used for learning. The region and legal jurisdiction in which your data is stored may also be controlled through your contract or regional settings. However, if you start a PoC without understanding these usages and send data containing confidential information to an external service, there is a good chance that the security department will order you to stop it.
Figure 2. Data flow comparison between cloud AI (left) and on-premises/local LLM (right). In the diagram on the left, data from PCs, smartphones, and servers is sent to a cloud outside the company, creating a risk of leakage. In the diagram on the right, processing is completed on a local server inside the firewall, and no data is exported.
Particularly in industries such as manufacturing and finance, where non-disclosure agreements (NDA) with business partners are strict, sending data to external services may violate confidentiality agreements and trade secret management, so it is essential to confirm the contract contents and operational design in advance.
Workaround
At the PoC planning stage, please include the information security department (or equivalent person) as a project member. You should classify the sensitivity level of the data you use, and when handling highly confidential data, consider adopting a local LLM (language model that runs on-premises) instead of a cloud API. However, while local LLM can reduce the risk of external transmission, it is important to remember that operational responsibilities such as authority management, log management, model updates, vulnerability response, and audit trails will be transferred to your company. In our local LLM implementation support, we always classify the data as confidential and organize the scope of operational responsibility at the first hearing.
Pitfall 4: Field resistance and change management
Even if PoC is technically successful, it is meaningless if it is not used in the field. Vague anxiety that “AI will take away my job”, inertia that “the old way of doing things is fine”, and lack of education that “I don't know how to use it”. When these three factors are combined, the AI system that has been built with great effort will not be used by anyone and the budget will be used up.
IPA's "DX Trends 2024" also highlights securing human resources and reforming corporate culture and climate as important themes in promoting DX. BCG's “10-20-70 Principles” also state that when it comes to AI value creation, algorithms account for 10%, technology and data account for 20%, and the remaining 70% involves the transformation of people and processes, which is a problem that cannot be solved by technology alone.
Workaround
Involve field members who will actually use AI in the project from the initial stage of implementation. The trick for PoC is to choose work that the field feels is “troublesome.” If people at the workplace feel that it will make a tedious task easier, they will voluntarily start using AI. If you simply order from the top down to use AI, it will end up being just a token use.
Pitfall 5: Undesigned operational system
The PoC will be completed in 3 months, but the operation will continue for many years. Upgrading the model, adding/updating data, improving prompts, responding to user feedback, and reverting in the event of a failure - it is necessary to design at the time of PoC who will maintain these, under what system, and with what budget.
In the case of cloud AI, there is also a risk that the pay-as-you-go API fee will be higher than expected. In the PoC, we will try it out with a small amount of test data, but if we deploy it company-wide, usage will increase significantly. Since costs vary greatly depending on the model used and preconditions (number of tokens, number of users, call frequency), it is important to calculate costs at the PoC stage based on the expected usage volume during company-wide deployment.
Workaround
Calculate the cost of the operation phase at the same time as PoC. In particular, for API usage fees, you should estimate the monthly cost by multiplying the expected usage amount when deploying company-wide by the unit price. If the cost exceeds the allowable range, on-premises operation using local LLM is an option, provided you have the capacity for frequent use, long-term operation, and in-house operation. Initial GPU procurement and operating personnel costs are required, but the greater the usage, the easier it is to reduce running costs.
Figure 3. Image of cost trends for cloud API pay-as-you-go vs. local LLM fixed costs (horizontal axis: number of months). Local LLM has a large GPU procurement cost (far left) at the beginning, but as the months go by, maintenance and operation costs (middle: gear) and personnel costs (right: people) become more important, and if usage is large, the cumulative charge can be lower than that of cloud API. The actual break-even point varies depending on the model scale, usage volume, and operational structure, so it is necessary to compare TCO.
A bird’s-eye view of the five pitfalls: Common root causes
What these five pitfalls have in common is that they confuse "technology implementation" with "business transformation." Generative AI is just a tool, and simply introducing a tool will not change your business. Redesigning business processes, changing people's mindset, and building an operational system -- only when these things work together can the PoC proceed to actual operation.
| Pitfall | root cause | Key actions for avoidance |
|---|---|---|
| 1. Underestimating data preparation | Illusion of company-wide implementation | Narrow down your target business to one |
| 2. Lack of ROI design | Expectations without quantitative indicators | Baseline measurement before PoC |
| 3. Putting security on the back burner | Technology lead/governance follow-up | Participation in security department at early stage |
| 4. Resistance in the field | top-down bias | Start with the “troublesome work” on site |
| 5. Undesigned operational system | PoC ≠ Lack of awareness of actual production | Incorporate operational cost estimation into PoC |
Why small and medium-sized businesses are advantageous
After reading this far, you may feel that it is only possible for large companies to do this, but in reality, small and medium-sized companies have some advantages. Decision-making is quick, communication with the front lines is easy because there are few organizational layers, and initial investment can be kept down because the scale of company-wide deployment is small. It is not uncommon for small and medium-sized enterprises to be able to get up and running in one month, whereas large companies take half a year to make internal adjustments.
The important thing is not to aim for perfection from the beginning. Start small with one business, one team, and one use case, and expand laterally when you see results. This approach is the best way to maximize effectiveness with limited resources.
Summary: List of things to do before PoC
Finally, I will organize the contents of this article as a checklist before starting PoC.
- Have you narrowed down your target business to one and clarified the data range for that business?
- Have you recorded current business indicators (processing time, error rate, etc.) as a baseline?
- Have you defined success criteria using quantitative KPIs?
- Have you classified the sensitivity level of your data and involved your security officer?
- Are field members participating in the project?
- Have you estimated the structure and cost (API fee or local LLM fixed cost) of the operation phase?
Just by knowing these things in advance, the probability of getting stuck in PoC will be greatly reduced. The introduction of generative AI is an initiative that can yield sufficient results even for small and medium-sized businesses if it is started in the right order and within the right scope.
Please contact us
LinkT&B provides one-stop support from considering the introduction of generative AI to PoC design and building and operating local LLM. You can consult with us even at the stage of “I don't know where to start” or “I started PoC but I'm stuck.” Please visit the IT & AI Solutions page or contact us directly from Contact us.
Reference materials
- Ministry of Internal Affairs and Communications “Reiwa 6th Edition Information and Communications White Paper”
- BCG「From Potential to Profit with GenAI（2024）」
- IPA “DX Trends 2024”
- OpenAI「How your data is used to improve model performance」
- Microsoft “Azure OpenAI Service Data Privacy”
- Google Cloud “Gemini API Data Governance”
Related Articles
Deploying Local LLMs with One Delivery Partner ／ Four Structural Risks in Cloud AI: Rethinking Encryption, VPNs and SaaS ／ Local LLMs: Addressing Cloud AI Risks Through Deployment Architecture ／ How Safe Is Cloud AI Data? The Case for Local LLMs
Further reading: NIST: Generative AI Profile
Discuss your project
We help select an AI architecture, evaluation process and operating model for your workflows and information requirements.