Samiul Alam

Hi there! I am an incoming PhD. student at Ohio State University. I work on improving privacy and fairness in deep learning. I used to work at Samsung Research and Development Institute in Bangladesh as a Software Engineer till 2021. I graduated from Bangladesh University of Engineering and Technology in 2017.

I am one of the co-founders of bengali.ai. I lead the collection and standardisation efforts behind popular open source Bengali Natural Language and OCR Datasets like NumtaDB, Bengali Graphemes and Common Voice Bengali Speech Dataset.

I am deeply passionate about my work and regularly try and find novel applications. As such, I have worked in a wide range of applications. If you have any questions about my research or have any relevant ideas in mind, feel free to e-mail me.

news

May 17, 2024	My work on multi-state filtering has been published in Ploss One.
May 12, 2024	Our survey paper on Efficient LLMs has been accepted at TMLR!
Dec 14, 2023	My research on Multi-channel Skin Conductance deconvolution was published at IEEE Journal of Open Engineering, Medicine and Biology
Aug 15, 2023	Finished my MS degree at MSU and joined OSU as a Graduate Fellow.
Jan 1, 2023	Humbled to be awarded the highly competitive OSU Graduate School University Fellowship!

selected publications

Efficient Large Language Models: A Survey

Wan, Zhongwei, Wang, Xin, Liu, Che, Alam, Samiul, Zheng, Yu, Liu, Jiachen, Qu, Zhongnan, Yan, Shen, Zhu, Yi, Zhang, Quanlu, Chowdhury, Mosharaf, and Zhang, Mi

Transactions on Machine Learning Research 2024

Abs arXiv PDF

Large Language Models (LLMs) have demonstrated remarkable capabilities in important tasks such as natural language understanding, language generation, and complex reasoning and have the potential to make a substantial impact on our society. Such capabilities, however, come with the considerable resources they demand, highlighting the strong need to develop effective techniques for addressing their efficiency challenges. In this survey, we provide a systematic and comprehensive review of efficient LLMs research. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient LLMs topics from model-centric, data-centric, and framework-centric perspective, respectively. We have also created a GitHub repository where we compile the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey, this https URL, and will actively maintain this repository and incorporate new research as it emerges. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of the research developments in efficient LLMs and inspire them to contribute to this important and exciting field.
Sparse Multichannel Decomposition of Electrodermal Activity With Physiological Priors

Alam, Samiul, Amin, Md. Rafiul, and Faghih, Rose T.

IEEE Open Journal of Engineering in Medicine and Biology 2023

Abs PDF

Goal: Inferring autonomous nervous system (ANS) activity is a challenging issue and has critical applications in stress regulation. Sweat secretions caused by ANS activity influence the electrical conductance of the skin. Therefore, the variations in skin conductance (SC) measurements reflect the sudomotor nerve activity (SMNA) and can be used to infer the underlying ANS activity. These variations are strongly correlated with emotional arousal as well as thermoregulation. However, accurately recovering ANS activity and the corresponding state-space system from a single channel signal is difficult due to artifacts introduced by measurement noise. To minimize the impact of noise on inferring ANS activity, we utilize multiple channels of SC data. Methods: We model skin conductance using a second-order differential equation incorporating a time-shifted sparse impulse train input in combination with independent cubic basis spline functions. Finally, we develop a block coordinate descent method for SC signal decomposition by employing a generalized cross-validation sparse recovery approach while including physiological priors. Results: We analyze the experimental data to validate the performance of the proposed algorithm. We demonstrate its capacity to recover the ANS activations, the underlying physiological system parameters, and both tonic and phasic components. Finally, we present an overview of the algorithm’s comparative performance under varying conditions and configurations to substantiate its ability to accurately model ANS activity. Our results show that our algorithm performs better in terms of multiple metrics like noise performance, AUC score, the goodness of fit of reconstructed signal, and lower missing impulses compared with the single channel decomposition approach. Conclusion: In this study, we highlight the challenges and benefits of concurrent decomposition and deconvolution of multichannel SC signals.
OOD-Speech: A Large Bengali Speech Recognition Dataset for Out-of-Distribution Benchmarking

Rakib, Fazle Rabbi, Dip, Souhardya Saha, Alam, Samiul, Tasnim, Nazia, Shihab, Md. Istiak Hossain, Ansary, Md. Nazmuddoha, Hossen, Syed Mobassir, Meghla, Marsia Haque, Mamun, Mamunur, Sadeque, Farig, Chowdhury, Sayma Sultana, Reasat, Tahsin, Sushmit, Asif, and Humayun, Ahmed Imtiaz

In Proc. INTERSPEECH 2023 2023

arXiv
FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction

Alam, Samiul, Liu, Luyang, Yan, Ming, and Zhang, Mi

In Advances in Neural Information Processing Systems 2022

Abs arXiv HTML PDF

Most cross-device federated learning (FL) studies focus on the model-homogeneous setting where the global server model and local client models are identical. However, such constraint not only excludes low-end clients who would otherwise make unique contributions to model training but also restrains clients from training large models due to on-device resource bottlenecks. In this work, we propose FedRolex, a partial training (PT)-based approach that enables model-heterogeneous FL and can train a global server model larger than the largest client model. At its core, FedRolex employs a rolling sub-model extraction scheme that allows different parts of the global server model to be evenly trained, which mitigates the client drift induced by the inconsistency between individual client models and server model architectures. Empirically, we show that FedRolex outperforms state-of-the-art PT-based model-heterogeneous FL methods (e.g. Federated Dropout) and reduces the gap between model-heterogeneous and model-homogeneous FL, especially under the large-model large-dataset regime. In addition, we provide theoretical statistical analysis on its advantage over Federated Dropout. Lastly, we evaluate FedRolex on an emulated real-world device distribution to show that FedRolex can enhance the inclusiveness of FL and boost the performance of low-end devices that would otherwise not benefit from FL. Our code is available at: https://github.com/AIoT-MLSys-Lab/FedRolex.
Bengali Common Voice Speech Dataset for Automatic Speech Recognition

Alam, Samiul, Sushmit, Asif, Abdullah, Zaowad, Nakkhatra, Shahrin, Ansary, MD, Hossen, Syed Mobassir, Mehnaz, Sazia Morshed, Reasat, Tahsin, and Humayun, Ahmed Imtiaz

arXiv preprint arXiv:2206.14053 2022

Abs arXiv HTML

Bengali is one of the most spoken languages in the world with over 300 million speakers globally. Despite its popularity, research into the development of Bengali speech recognition systems is hindered due to the lack of diverse open-source datasets. As a way forward, we have crowdsourced the Bengali Common Voice Speech Dataset, which is a sentence-level automatic speech recognition corpus. Collected on the Mozilla Common Voice platform, the dataset is part of an ongoing campaign that has led to the collection of over 400 hours of data in 2 months and is growing rapidly. Our analysis shows that this dataset has more speaker, phoneme, and environmental diversity compared to the OpenSLR Bengali ASR dataset, the largest existing open-source Bengali speech dataset. We present insights obtained from the dataset and discuss key linguistic challenges that need to be addressed in future versions. Additionally, we report the current performance of a few Automatic Speech Recognition (ASR) algorithms and set a benchmark for future research.
A large multi-target dataset of common bengali handwritten graphemes

Alam, Samiul, Reasat, Tahsin, Sushmit, Asif Shahriyar, Siddique, Sadi Mohammad, Rahman, Fuad, Hasan, Mahady, and Humayun, Ahmed Imtiaz

2021

Abs arXiv HTML PDF

Latin has historically led the state-of-the-art in handwritten optical character recognition (OCR) research. Adapting existing systems from Latin to alpha-syllabary languages is particularly challenging due to a sharp contrast between their orthographies. Due to a cursive writing system and frequent use of diacritics, the segmentation and/or alignment of graphical constituents with corresponding characters becomes significantly convoluted. We propose a labeling scheme based on graphemes (linguistic segments of word formation) that makes segmentation inside alpha-syllabary words linear and present the first dataset of Bengali handwritten graphemes that are commonly used in everyday context. The dataset contains 411k curated samples of 1295 unique commonly used Bengali graphemes. Additionally, the test set contains 900 uncommon Bengali graphemes for out of dictionary performance evaluation. The dataset is open-sourced as a part of a public Handwritten Grapheme Classification Challenge on Kaggle to benchmark vision algorithms for multi-target grapheme classification. The unique graphemes present in this dataset are selected based on commonality in the Google Bengali ASR corpus. From competition proceedings, we see that deep learning methods can generalize to a large span of out of dictionary graphemes which are absent during training (Kaggle Competition kaggle.com/c/bengaliai-cv19, Supplementary materials and Appendix https://github.com/AhmedImtiazPrio/ICDAR2021supplementary).
Numtadb-assembled bengali handwritten digits

Alam, Samiul, Reasat, Tahsin, Doha, Rashed Mohammad, and Humayun, Ahmed Imtiaz

arXiv preprint arXiv:1806.02452 2018

Abs arXiv HTML PDF

To benchmark Bengali digit recognition algorithms, a large publicly available dataset is required which is free from biases originating from geographical location, gender, and age. With this aim in mind, NumtaDB, a dataset consisting of more than 85,000 images of hand-written Bengali digits, has been assembled. This paper documents the collection and curation process of numerals along with the salient statistics of the dataset.