A team of students develops Artificial Intelligence algorithms to digitally reconstruct the damaged murals from Ajanta Caves. Chahak Mittal talks to them about how AI can help historians solve ancient puzzles
Imagine walking inside the oldest fort or monument in your city. You find out that the building is in a tattered condition. All that remain are blackened corners, broken window panes, crooked steps, faded marble floors, stones chiselled out of the walls, and tarnished inscriptions. Now imagine viewing everything in the condition it would have ideally been when the rulers living in that palace actually existed. Well, this is no more a mere thought or a wishful imagination, if we think digitally!
Reconstructing places of historical significance physically might include a lot of archaeological investigations, primary and secondary sources of research, guesswork, labour, funds and time. However, ever imagined if we were to revamp and reconstruct them digitally?
Well, Ancient AI, a team of IIT Roorkee students, has developed deep-learning Artificial Intelligence (AI) and Machine Learning (ML) algorithms to digitally provide aid to the restoration process of murals recovered from the Ajanta Caves in Maharashtra. The restored digital murals have now been deposited at the Arctic Digital Archive (AWA) located close to the North Pole. This will ensure its protection from natural disasters and secure accessibility for future generations for over 1,000 years.
So how does the algorithm work? Parth Chhabra, second year student of Mechanical Engineering and leader of the team, says, “The algorithm that we have created helps fill the spots and spaces present in an image based on their surroundings and previously-learned features from various other paintings. Also, we can use it to colour the image to give it a more realistic look.”
The algorithm was worked upon as a part of the Tech4Heritage Hackathon, which aimed at finding AI & ML-based solutions to help in the restoration process of murals recovered from the Ajanta Caves. It was spearheaded by the Rishihood University, Internity Foundation and Sapio Analytics, supported by the Ministry of Culture and LTI (Larsen & Toubro Infotech). Ancient AI, the winner team, now has the opportunity to work closely with the Sapio Analytics Heritage Restoration Team.
Another student from the team, Kushagra Babbar highlights that they utilised a dataset of more than a 1,000 reference paintings to develop AI models built on General Adversarial Networks (GANs). So what were the challenges in curating the data? He adds, “The biggest challenge that we faced was creating a data set of damaged paintings and their corresponding restored images, which are not readily available. Hence, we thought of a completely opposite approach, that of damaging some clean images to an extent, which matches real-life damaged images. With image-processing, we took spots from the damaged ones and overlaid them on the clean ones. Post that, through Neural Style Transfer, a fascinating AI achievement, we styled our images similar to the damaged ones. With this and some randomisation, we created a dataset of approximately 850 damaged images from the 36 clean ones, of the paintings. It took around 40 to 50 hours generating this much big dataset. Completing this step was very crucial since our result for the next step depended a lot on this. It was a very challenging process and this could not have been done without team work. We divided the images equally among us and generated 210 images each.”
Several museums and exhibitions are now evidence of how art is being displayed through AI and VR technologies, making experts believe that AI is the future face of preserving history. Sahil Agarwal, co-founder and CEO, Rishihood University, explains how, “Technology certainly has a massive role in the future of art and history. There are several benefits to this. Today, when I enter the fossils section of a museum, I can not only see the part of an animal’s skeleton, I can visualise the real animal movement and other characteristics such as how it grew older and so on, through the AI. When I look at a painting, I can feel as if the painting is a visual reality around me. People are developing games and videos on history using AR and VR. Similarly, AI can be used to curate, store and recreate art. I believe the day is not far where the machine can learn from an artists’ style and recreate an art.”
So, what would an art on climate change look like if Van Gogh were to depict it in his style? he questions and says, “I think AI will reach that sophistication!”
Finding out evidence for historical theories and tracking records of past events have only been hindered by the labour-intensive process of identifying data from artefacts and handwritten inscriptions. Talking about how AI can help historians solve ancient puzzles, Kushagra says, “AI is the most efficacious way to build any automated solution. With the help of different algorithms in computer vision and natural language-processing, we can extract and even predict the ignored and missing information about ancient civilisations. Even if it is an old painting or an ancient literature in an unknown language, through AI, we can restore and know our past much better. I think the hackathon was a great opportunity to channelise our skills and have a head start in restoring our own culture, identity and heritage.”
Even though the technique might be of major help to the historians, experts argue that looking at the current pandemic and frequently-occurring health haphazards, archaeological puzzles do not seem to be as urgent as computer science projects in healthcare, finance and other industries. Talking about whether AI’s usage has more of an urgency towards healthcare infrastructure rather than historical research at present, Sahil says, “Algorithm-based decision-making is quite useful, definitely in the areas such as healthcare, where we are short of skilled professionals. However, I will not say that there is an urgency because the robustness and efficiency of such technologies develop over time. They need constant human supervision to maintain and improve their decision-making. The implication of incorrect results from technology in healthcare is much more severe than in an area like archaeology. Therefore, we should not prioritise among the two. Rather, we should judge whether the technology has sufficiently matured so that it can be applied for mass usage in a particular area.”
Parth agrees that applying algorithmic techniques to historical research can improve AI’s capabilities “more than directly applying them to a sensitive area like healthcare right now.”
Well, why just healthcare and finance, Sahil believes that “education” is an area where AI has immense potential. He adds, “It can become a 24x7 companion for a student. Using AI, a person can realise her/his current levels and ways of learning, how efficiently one reads and what time of the day is best suited for one to read, what skills one needs to pick up if s/he wants a particular job, etc. Similarly, employers can use AI to track the levels and speed of learning much before the recruitment and for on-the-job training. Some companies are developing AI-based recruitment models, too.”
For Parth, “AI can excel in Agriculture and Farming, Security and Surveillance, Sports Analytics and Activities, Manufacturing and Production, Live Stock and Inventory Management, Autonomous Vehicles, Logistic Supply Chain, creating Virtual Assistant or Chatbots, Retail and Fashion.”
He adds that while this was about recreating the images of the damaged structures, the team aims to work on a project that could involve “interpreting the ancient texts through AI!”
