TY  - JOUR
T1  - Construction of Malay Abbreviation Corpus Based on Social Media Data
AU - Omar, Nasiroh AU - Farhan Hamsani, Ahmad AU - Atiqah Sia Abdullah, Nur AU - Zaleha Zainal Abidin, Siti 
JO  - Journal of Engineering and Applied Sciences
VL  - 12
IS  - 3
SP  - 468
EP  - 474
PY  - 2017
DA  - 2001/08/19
SN  - 1816-949x
DO  - jeasci.2017.468.474
UR  - https://makhillpublications.co/view-article.php?doi=jeasci.2017.468.474
KW  - Social media
KW  -user generated content
KW  -abbreviation
KW  -corpus
KW  -big data
AB  - This study describes a construction of Malay abbreviation corpus by extracting and normalizing
selected social media data with multilayer filtration pattern matching technique along with statistical machine
translation approach. In this study, one million Malay Lingo user-generated-posts via Twitter and Facebook
are extracted for sampling. Each word will undergo pre-processing stage which involves filtration and
association and stored in MySQL database table. Then, each word in the corpus is linked with its respective
word in existing vocabulary; otherwise, it is considered as abbreviation word will be further processed by using
N-grams approach and added to the existing corpus. Based on the result, it can be seen that the longer the
length of text, the translation probability is decreased. Furthermore, the style of writing is very important. The
lack of space usage to separate in between words will cause more than one word are merged and
became out-of-vocabulary word. The worst case is the strange merged word has no link to any recognizable
root word in the dictionary. In the first attempt of processing 1000 selected posts from the social media, a lot
of uncommon abbreviation words are found. As a result, a lower translation percentage is achieved.
Nevertheless when the post uses common abbreviations that exist in the Malay Social Media Corpus then the
result of the translation is able to achieve 100% accuracy. Nevertheless, the source of user-generated word is
infinite and there is still many ways to improve the combination of NLP techniques in constructing a better and
reliable corpus due to the dynamic nature of user’s behaviour and their informal ways of writing texts. The
corpus is very much needed in analysing public’s sentiments in various dimensions such as product-related
evaluations and service-oriented feedbacks which are propagated across various platforms of social media.
ER  -