Prompt designing and prompt tuning have marked a distinct milestone in NLP research with the advent of Large Language Models (LLMs). Large-scale pretraining allows these models to process natural language inputs for any downstream tasks and then map the output labels as generated tokens. However, the design of optimal prompts is still an open question, both in a fine-tuning setting (full/partial model parameters are updated through backpropagation in the downstream task) or in-context-learning (no gradient update). Our findings along with multiple contemporary endeavors on different NLP tasks suggest that alignment plays a crucial role in prompt designing. First, one needs an alignment between the pretraining objectives and the downstream task. Second, one can impose suitable heuristics in example selection and prompt-template design to significantly improve the in-context learning abilities of LLMs: both in mono-lingual as well as cross-lingual settings.

Tanmoy Chakraborty is currently an Associate Professor of Electrical Engineering and an Associate Faculty of the Yardi School of Artificial Intelligence at the Indian Institute of Technology Delhi. Prior to this, he was an Associate Professor of Computer Science at IIIT Delhi, where he was also the head of the Infosys Centre for AI. In 2018, he spent at the Max Planck Institute for Software Systems, Saarbrücken, Germany as a DAAD Visiting Faculty. He leads the Laboratory for Computational Social Systems (LCS2), a research group that broadly works in the areas of Natural Language Processing, Computational Social Science and Machine Learning. His current recent interests include designing explainable and tiny language models for text and multimodal contents, conversational AI, and graph neural networks. Tanmoy did his PhD from IIT Kharagpur in 2015 as a Google PhD scholar and worked as a postdoctoral researcher at the University of Maryland, College Park. He is a recipient of several awards/recognitions such as the Ramanujan Fellowship, several faculty awards/gifts/grants (from Google, Accenture, LinkedIn, and TensorFlow), the PAKDD'22 Early Career Award, IEI Young Engineers Award 2021-22 and Paired Indo-German Early Career Award 2023. He is the author of two books -- "Social Network Analysis" (a textbook), and "Data Science for Fake News". More details may be found at tanmoychak.com.