As a human worker, it bothers you when you read, watch, or hear stories that your job could get replaced by a robot or an AI program soon. Even I got a bit worried when I heard about Devin, the first AI software engineer and my initial reaction was that this would never happen, but I may be wrong. AI has the potential to eventually cannibalize the information technology industry.
On the other hand, an employer or any other business entity appreciates the idea of using AI and likes to explore options simply because they want to reduce human dependencies by automating their jobs. They are very well aware that automated jobs are faster, run 24x7, are more efficient, and are much cheaper, above all, automation comes with no emotional baggage.
As we talk today, humans are already being replaced by AI. Now, if anyone loses a job to AI, who will be taking his/her salary? The AI program has nothing to do with salary. A driverless car is not making money for an AI driver, so eventually, the bucks stop at a human entity.
As a technologist, what should I/we do now? Should we wait for an AI program to take over our jobs or should we try to jump to the other side of the industry and be a perpetrator instead of a victim?
Automating a manual task
Achieving 100% automation using AI is not new in the industry, however, the cost of such automation was significantly higher in the past than what it is today. Building custom ML models was a costly affair as it needs a lot of data and training, top-notch programmers and there is always a risk of the model not performing up to the mark. But now, with ready-to-use technology like Open AI, you can achieve automation easily and with absolutely no AI background.
For the past couple of years, I have been working closely with various back-office teams trying to automate their day-to-day job. Be it invoicing or claims processing, underwriting, or loan processing, the approach towards automating such tasks is now well-defined and mature. In simple terms, we introduce various software tools & technologies (off the shelf or custom) into their daily job routine work and try to improve their productivity and efficiency.
Before the breakthrough innovation and easy availability of technologies like GEN AI, I was somehow convinced that with a decent budget, you could automate a back office process up to a certain extent and you would reach a threshold where human interventions are required, however now I have realized that the next couple of years will be the game changer for the human workforce, especially operations and back office where hyper-automation may take away most of the work which they are performing today.
Profile Screening Example
Let me take a simple example of how we can introduce automation step by step and finally try to make a manual process human-free or require minimal human interventions.
In the example below, a human employee typically an HR has to read & analyze candidate profiles which they receive daily via emails or some other source. Their job is to read emails, download attachments, perform indexing /classification on the profiles, and finally analyze if the candidate is fit for the job as per job descriptions provided to them or by checking existing open positions available in the organization. This profile screening use case could be applied to any other scenario for example invoice, claims processing, loan processing, verifications etc.
Manual Process
First, let us visualize how a human worker could perform this function without any tool or application.
Assuming a common mailbox is available for all users where profiles are collated, human workers will first read emails. Once read, he/she will first download the profile and scan it for its authenticity, once done he/she will start screening the profile, classify it, and try to match it with the company requirement/job description.
He/ She could accept or reject a profile, if a profile is accepted, an HR person will try to read details like demographic, educational qualifications, experience, etc., and manually punch these details into some system of records, like their company's HR system.
Now one word of caution before we proceed further. Trying to automate a business process should never be driven by your enthusiasm to implement cutting-edge technologies like Open AI or peer organization pressure. Attempting automation especially using AI should be driven by a strong business case, ROI, technology roadmap, and future business scaling prospects.
Whenever I approach any automation problem as above, I always suggest my stakeholders take a step-by-step iterative approach, especially for attempting AI-based automation from scratch. However, I find it difficult to convince most of them simply because of the information overload they carry with them. Everyone expects magic overnight and that's what is the recipe for failure.
So why an incremental approach? Simply because it will de-risk your existing process and secondly taking a big bang approach is something I never recommend, factors being big investment risk, long GTM cycle, difficulty in rolling back, etc.
Returning to our screening example, let's get a view of different activities a human user is performing and see what activities we can automate easily using a non-AI-based program.
As you can figure out on your own, we can easily automate tasks like reading emails, downloading and extracting documents(profiles), and storing information in a system of records.
Level#1 Automation - Human in the loop
Now the above automation can easily be achieved using custom-developed applications or prebuilt platforms which can help in automatically reading mailboxes, extracting contents using OCR and extraction tools, and finally publishing data by integrating into a downstream system.
Amazingly Open AI can also be used to extract information from a document like a candidate profile by writing simple prompts, it is just that you may need to perform OCR if the contents are PDF or image files.
Below are some prompts examples that can be used to extract information from a profile document.
If you notice, with simple prompts, we can extract details like name, email, etc from any unstructured documents shown above, be it any available format. Now, there is a catch here If you have noticed, one additional human process needs to be added though, this is to validate the job that got automated by the system.
In the operations and back office world, especially with document processing, there is an interesting and common concept known as "Human In the Loop".
Typically, when a document is first digitized using OCR/Extraction technology, it is passed via a human user to validate the output to ensure that automation has done its job as expected and it also allows human users to perform necessary corrections if required.
Secondly, humans are also capable of making decisions quickly, in this example ensuring that this profile is relevant and suitable for the openings available in the organization, which we typically call this job description (JD).
With Level#1 automation, there is hardly any use of AI or ML (Except Open AI for document extraction which can be replaced by any other tool). Humans are still playing an important role in the process.
Level#1 automation will help in achieving considerable productivity improvements for human users, it could reduce your human task force by 30%-40%, but you still need human intervention for review and decision-making purposes.
Level#2 Replacing Humans with OPEN AI (Human Out of the loop)
Going beyond Level#1 automation requires a deep understanding of functions a human user is performing, especially reviews and decision-making. Such functions are difficult to automate and need an AI intervention for sure.
After level#1 automation of this use case, humans now perform functions like extraction and classification reviews, profile matching/screening (Accept/Reject), and summarizing contents.
We can very well create 3 ML models and try to achieve all of the above 3 functions using custom AI models, however doing this requires considerable time, data, engineers, and a full SDLC deployment lifecycle which is again painful and has its own set of drawbacks.
We can try to use Open AI to perform the above three decision-making and review functions. I will give you one example for each of such functions, however, the actual enterprise use case needs a more detailed analysis and design strategy.
With Open AI, this can be achieved very easily using their ready-to-use GPT models and prompt engineering features. Let us try to write prompts to achieve the below functions
Function#1 - Review/validate candidate information
Function#2 - Check if the candidate is suitable for a job or not (You can provide JD in the prompt itself which is called zero shot)
Function#3 - Summarize various information available in the resume.
There are again possibilities of Open AI not performing up to the mark or may go wrong, to handle such situations we can have human users validating fallout information/low confidence information processed Open AI. Your Open AI replacing humans in Level#2 may look like below.
Level#2 automation should be able to provide 70-80% of automation benefits. We still have 20% of human interventions required to get the job done.
Level#3 Automation - AI Out of the loop
Finally, let us try to automate our use case a bit further and see if it is possible to remove human dependency and achieve hyper-automation. Achieving Level#3 automation is a much more difficult, tedious, and incremental process.
Level#3 type of automation should be attempted only after Level#2 is in production and you have sufficient data to analyze and work on further. Let's think about what scenarios could be handled by Level#2 human staff.
I could think of fallouts like an invalid email address, and incomplete or missing information.
All these fallouts could be the result of AI inaccuracy, hallucinations, bad scanning, or low confidence scores. What is the resolution for that? Below is how the end state will look like once we try to enforce further automation
The resolution could be anything from auto-completion of missing information like state, based on city provided, auto-correction of email addresses, etc. There could be more scenarios like human users replying to candidates for more inputs or information.
It all depends upon the data we will gather after running Level#2 automation for a couple of months and performing further analysis on this data. You may also need to write custom routines/functions to ensure complete closure of the automation loop like writing auto response asking for more information. There is again a possibility of humans playing a role here since the other systems your application may be interacting with may not support automation.
I hope you have enjoyed the above article and I would love to hear direct feedback and suggestions from all of you.
Best Regards
Sachin Tah | Sr. Director - Technology | Cognizant Technology Solutions | USA
Comments