It is known for us that moslems were celebrating Eid Al Adha festival last Friday until Monday. The story behinds the festival has been foretold in the Holy Quran. Other scripture i.e. the Old Testament also mentions about the prophet Abraham pbuh sacrificed his only son (at that time), despite the dispute about which of prophet Abraham’s son that was going to be sacrificed. I won’t discussed about the dispute, but I will focus on the meaning of the holy festival.

Eid Al Adha is about sacrifice and moslems who are capable of doing sacrifice should sacrifice their beloved possessions to be offered to God. In Indonesia, we call Eid Al Adha festival as hari raya kurban. Kurban or qurban derived from arabic word qurb/qoroba means getting near or close. It means by doing sacrifice, moslems will be able to get closer to God and reduce their loves toward this deceptive world. However, there are three different perspectives that can be observed regarding the festival as follows.

Firstly, from the viewpoint of Human-God relation, Eid Al Adha means a total submission to the commands and will of God. We have to take example set by prophet Abraham in obeying command to sacrifice his precious son. Also, it means that people have to redeem themselves in order to be close to God by giving up their love of possessions and love of world. By getting closer to God, our behaviours/values will  be affected by God’s values such as just, forgiving, merciful, and clean. Thus, we can purify our heart which always lead us to remember God in the whole aspects of our daily lives. If we are truly having such personal integrity and values, I believe that the problems of corruption and collusion in our country will never occur.

Secondly, from the viewpoint of Human-Human relationship, by doing sacrifice means that we will find our humanity and its values. It will deprive our egos, encourage togetherness, and promote empathy and  sense of caring towards others  since we are sacrificing what we have to be given to others who are in need. Hence, the bond of brotherhood between us  will be strengthened and create a healthy social relationship.

Thirdly, from the standpoint of Human-Cosmos relationship, we have to realize our role in cosmic balance and our existence within. God creates human(us) to be caliph in this world so that we manage the world in harmonius order. Therefore, we  have to respect nature and treat nature in a good manner by sacrificing our comfort, possessions, etc that will keep the natural balance. Bike to work is one of good example in reducing CO emission to fight against global warming .

In sum, sacrifice does not mean that it has to be cattle or money, but it is more about the willingness to sacrifice our most precious thing in our life such as time, comfort, life style, etc. Therefore, in this holy festival, let us sacrifice something valuable in our life to make us a better man in front of God, humanity, and environment.

I have been looking for a PhD for quite some times. I read in a website that there was a vacancy as PhD student in a project related to video system and its architecture. The topic is very interesting since I have sound knowledge in multimedia, especially MPEG video codecs and video system. Hence, I applied for the position. Two weeks later, I got a notification that I was shortlisted via email. It said that I will be interviewed on 11-18-2009 via skype. I was very excited so that I couldn’t sleep well during that night. Four days prior to the scheduled interview, I opened my thesis and tried to remember my researches during my master. I read some papers related to embedded system and scheduling. I installed skype on my laptop and tried to use it from the campus, but the audio quality was very poor. Hence, I decided that I will go to an Internet cafe for the interview.

And the story continues..

On the scheduled day, I couldn’t think of any but the interview. I was pretty anxious yet excited at the same time. Exactly at 3.00 pm (GMT+7), the professor chatted with me and asked me to be prepared for the interview. The interview took about 40 minutes and some of the questions were not as I predicted before. Sometimes I couldn’t catch professor questions since it was pretty crowded in the Internet cafe and the music was pretty loud. Nonetheless, I have answered them according to my best knowledge and understanding. My regret is that there were some questions that I should have answered correctly, but I didn’t hit the target well. I should have confirmed the questions to the Professor instead of directly answer it. Well, regret always come late, right?  Yet, I will be positive regardless of the outcome. I won’t quit looking until I obtain a PhD with suitable topic. Wish the best for me.. (The results will be announced in two weeks from now ^_^ ganbatte..)

In the past three days the Oracle Server that is usually used by students for database practical course experienced  error. The error was that the listener couldn’t listen to any connection request from users to port 1522, though the TNSNAMES.ora and LISTENER.ora have been correctly configured. The error caused the Oracle DB Server inaccessible from any programming application such as NetBeans, JDeveloper, and Eclipse. Also, it made the Oracle EM cannot access the database instance. I have restarted the computer, the database instance, and the listener, but it resulted nothing. After three days of nothing, my colleague restarted the database instance and the listener and guess what,, it works again. Huh, I can’t understand what is the root cause of the problem. Nonetheless, I will definitely find it out if the problem happens again. Ganbatte!!

Taking one day break after working for three consecutive weeks is indeed very refreshing. I spent my time by swimming in Atlas, a sport center that I frequently visit, and then I had my car washed. My body feels so great n my mind feels so relax. In fact, I want to go to a health spa for a massage , but it was 5 pm already. Hence, I canceled it. In exchange, I went to a barber shop to have my hair cut. Starting from Monday and afterward, it will be a very hectic month of November. This is part of what Moslems called Jihad to fulfill the necessities of our family. Oops, I’m a single, so I guess I have to do Jihad for myself then

Life is never flat.. Work hard, Think smart, Save enough, Give much.. Rewarding now and hereafter..

Last week I conducted a written test for business intelligence (BI) course. This course is delivered 60% in English. The exam consists of 30 multiple choices and 5 essay questions. The exam material encompasses introduction to BI, BI system, a lil bit of data mining and its algorithms such as Decision Trees and artificial neural networks.

The problems that I gave were quite simple based on presentation slides that I made, but most of students couldn’t do almost all the test problems. Just a preview, the students could do at best 10 out of 30 multiple choices and 2 out of 5 essays. This is quite a surprise for me since I predict they will score around 65 on average.

So, who has to be blame for the bad result? Is it because the class uses English as instructional language? I don’t think so. Based on my observation during classes, the communication went very well. Is it because the problems were too hard? Nope, I gave them the middle level questions. How about the exam questions that fully in English? Hey, our reference books are in English, so it should not be a problem for them. Is it because my explanation was too fast to follow during classes? Hmm, perhaps I should review my teaching speed by asking the students in the upcoming week.

From the beginning of the class, most of the students consider BI as a tough course. Hence, they lack of confidence in attending the course. If I could just open their perspectives, I think they would have a better result. One more thing, they should study harder and do the experiments diligently. Otherwise, all encouragement that I gave them will mean nothing. I hope that in the next test, the result will be much better..

In the last two weeks I was eagerly seeking relevant PhD programs in European and Australian universities to continue my study. I have two plans that I called plan A and plan B. Plan A is that I’m applying to universities that provide me scholarships or research allowances. Hence, I must find the suitable research vacancies in various universities, which is not easy, Why? Because the applicants may come from any part of the world. Nonetheless, it’s still the easiest part of pursuing PhD, am I right?

Next, plan B is to apply to universities regardless of the availability of research allowances. Wait, how will I finance my study? The answer is that I’m planning to apply DIKTI scholarships from Indonesian government that are opened to lecturers in all universities in Indonesia. Thus, what I need is merely an acceptance letter from a foreign university. Yet, I have to find a Professor that is willing to supervise my PhD. It might look cumbersome, but it’s easier since plan B has higher degree of success and more option in determining which university we want to apply.

Now, one issue that often discussed by us is the ranking of university that we apply. Does the university have an excellent reputation? Is it well-known worldwide? I’m not going to be a hypocrite. I think it does affect our preference in selecting university we want to attend. However, it’s not the only thing that we should consider. The most essential part for us (Master/PhD seekers) is the research subjects that university has to offer could meet our research interest. In addition, the research facilities are sufficient to what we need and the research atmosphere should be conducive for study.

A short story,,, when I’m applying for my master degree, I got very lucky. I obtained my master degree from ICU which then merged into KAIST. KAIST is considered as one of the best universities in Asia (69^th in the world and 7^th in Asia according to THES-QS). Moreover, unlike other students that awarded scholarships from institutions or universities, my case was rather unique And again I got lucky. Thanks to God and thanks to my Professor, my colleagues, my friends! You guys are awesome. So without looking at the university ranking, I got myself lucky twice..

Okay, now back to finding PhD. I’m planning to conduct research in one of my three major interests: decision support system (data/text mining, machine learning, knowledge based system), multimedia (image/video coding and analysis), and database (xml technology for database, database technology). The first one is based on my current lab objectives and courses that I taught in ITS. The second one is based on my master degree’s research topic, whereas the third one is based on my bachelor degree’s research topic and course that I taught in ITS. Thus, my initial step would be making a list of universities that:

1. Offer suitable research subjects + research allowances/salary (Plan A)

2. Offer only suitable research subjects (Plan B)

Afterward, I sort them according to the most interesting research subjects and contact the Professors in charge. Hopefully I will make it to next round i.e. interview and admission in hand. Ow.. and about the university ranking, hmm.. It will be the last thing I will think about And I wish I will not have to think about Plan C hahaha..

In Part 2, we have delivered how we train the networks in ANN. This part will cover the last series of neural network in BI i.e. building ANN application and its application example. First, lets discuss about the steps in building ANN application. The steps can be explained concisely as follows.

1. Step 1 (Data collection): The data to be used for the training and testing of the network are collected. The important considerations in this step are that the particular problem is amenable to neural network solution and that adequate data exist and can be obtained.
2. Step 2 (Training and testing data separation): Training data must be identified, and a plan must be made for testing the networks’ performance. In this step, the available data set is divided into training and testing data sets. For a moderately sized data set, 80% of the data are randomly selected for training, 10% for testing, and 10% secondary testing.
3. Step 3 (Network architecture): A network architecture and a learning method are selected. The important considerations are the exact number of nodes and the number of layers.
4. Step 4 (Parameter tuning and weight initialization): There are parameters for tuning the network to the desired learning performance level. Part of this step is initialization of the network weights and parameters, followed by modification of the parameters as training performance feedback is received. Often, the initial values are important in determining the effectiveness and length of training (related to convergence and local minima problem).
5. Step 5 (Data transformation): Transforms the application data into the type and format required by the ANN. For example, transform sex attribute values from Male/Female into 0/1.
6. Step 6 (Training): Training is conducted iteratively by presenting input and desired or known output data to the ANN. The ANN computes the outputs and adjusts the weights until the computed outputs are within an acceptable tolerance of the known outputs for the input cases.
7. Step 7 (Testing): Once the training has been completed, it is necessary to test the network. The testing examines the performance of the network using the derived weights by measuring the ability  (accuracy) of the network to classify the testing data correctly.
8. Step 8 (Implementation): Now a stable set of weights are obtained. The network is ready to use as a stand-alone system or as part of another software system where new input data will be presented to it and its output will be a recommended decision.

Next, lets take one practical application of ANN i.e. bankruptcy prediction (taken from R.L. Wilson and R. Sharda, “Bankruptcy Prediction Using Neural Networks,” Decision Support Systems, Vol. 11, No. 5, June 1994, pp. 545-557). The bankruptcy prediction system can be described below.

Application Design

The system has five inputs which correspond to five well-known financial ratios i.e.

1. X1:  Working capital/total assets
2. X2: Retained earnings/total assets
3. X3: Earnings before interest and taxes/total assets
4. X4: Market value of equity/total debt
5. X5: Sales/total assets

In the output, the system has one output node that gives the final classification showing whether the input data for a given firm indicated a potential bankruptcy (0) or nonbankruptcy (1). The data source consists of financial ratios for firms that did or did not go bankrupt between 1975 and 1982.

The ANN Architecture

The ANN architecture can be shown in Figure 1.

Figure 1. ANN Architecture for Bankruptcy Prediction

Training

The data set, consisting of 129 firms, was partitioned into a training set and a test set. The training set of 74 firms consisted of 38 that went bankrupt and 36 that did not. The needed ratios were computed and stored in the input file to the neural network and in a file for a conventional discriminant analysis program for comparison of the two techniques. In addition, the neural network has three important parameters to be set: learning threshold, learning rate, and momentum.

Testing

The neural network was tested in two ways: by using the test data set and by comparison with discriminant analysis. The test set consisted of 27 bankrupt and 28 non-bankrupt firms. The neural network was able to correctly predict 81.5% of the bankrupt cases and 82.1% of the non-bankrupt cases i.e.

1. Overall, the ANN did much better predicting 22 out of the 27 actual cases (the discriminant analysis predicted only 16 cases correctly).
2. An analysis of the errors showed that 5 of the bankrupt firms classified as non-bankrupt were also misclassified by the discriminant analysis method. A similar situation occurred for the non-bankrupt cases.

The paper explained that the result of the testing showed that neural network implementation is at least as good as the conventional approach. An accuracy of about 80% is usually acceptable for ANN applications. At this level, a system is useful because it automatically identifies problem situations for further analysis by a human expert.

This writing  concludes the series of Neural Network in Business Intelligence. Hopefully it will be beneficial for you in learning the neural networks and its application in real life.

As I have discussed in Part 1, that neural networks (NN) are used to enhance business decision by imitating the brain mechanism in processing information. Figure 1 exhibits the comparison between biological neuron and artificial neuron in term of  elements that form them.

Figure 1. Biological vs Artificial NN

In processing information, there are five important parts in Artificial Neural Network (ANN):

1. Input which each input node corresponds to a single attribute e.g. for loan application, some applicant attributes can be age, income level, and home ownership. Some available data need to be preprocessed into meaningful inputted data with symbolic representation or scaling e.g. transform non numerical attributes into numerical attributes
2. Output which is solution to the problem e.g. in load application, the output can be yes or no with numeric values. Output needs to be post-processed. For instance, rounding output values into 0 or 1.
3. Connection weights which is the key elements of ANN. It express relative strength (importance) of input data or many connections that transfer data from layer to layer. In here, weights store learned patterns of information through repeated adjustments of weights.
4. Summation function which computes the weighted sums of all the input elements entering each Processing Element (Node), see Figure 2.
5. Transformation (transfer) function which is in form of activation level function. It can be linear or linear function. For a linearly separable data, a linear or discrete function may be applicable. However, linearly inseparable data should use non-linear function such as Sigmoid function or Tangential function (see Figure 3). This transfer function will normalize output values into reasonable values.

Figure 2. Summation Function

Figure 3. Sigmoid Function as One of Transfer Function

Training the networks can be through supervised learning and unsupervised learning. For the differences between those learning, please read my previous post: Introduction to Machine Learning. In here, we will only discuss about training the networks through supervised learning. The most prominence algorithm in training ANN is Back-Propagation (BP) algorithm based on Iterative Gradient Descent that wants to find the best weights combination that minimizes certain error function  (function of weights) by rolling down the error slope (Figure 4). The summarized algorithm can be shown in Figure 5 below.

Figure 4. Gradient Descent by Rolling Down The Error Slope

Figure 5. Summarized Iterative Gradient Descent Algorithm

In training the weights using BP algorithm, we want to minimize an error function which is the sum of squared error between the desired output and the actual output using non-linear transfer function. The weight update rule can be shown below.

Given

1. delta (error) = (DO(j) – AO(j))(df/dx), where DO(j) and AO(j) denotes desired output and actual output of node number j in output layer, respectively, and f is a transfer function.
2. f=[1+exp(-∑WiXi)]-1, where Xi and Wi denote input node number and weight of from input-i to the hidden layer, respectively.
3. df/dx=f(1-f) which is called logistic function to keep the error correction bounded.

The weights update rule is Wi(next) = Wi(prev) + alpha . (DO(j) – AO(j)) . f(1-f) . Xi,where alpha is learning rate that control the learning speed.

Note that too large alpha will lead to much correction results in going back & forth among possible weights values, never reaching the optimal point, whereas too small alpha may slow down the learning process.

The BP steps can be detailed as follows.

1. Initialize weights with random values and set other parameter
2. Read in the input vector & the desired output
3. Compute the actual output by working forward through the layers
4. Computer the error
5. Change the weights by working backward from the output layer through the hidden layers

Now, what is the problem of BP algorithm? BP algorithm does not guarantee that the weights will converge into global minima. In other word, it may be trapped in local minima (local best solution), especially if we have more than one minima. It depends on the initial weights that we assign. We are luckily enough if we give a good initial weights that corverging to global minima. There are several solution that can be adapted to cope with this problem:

1. Adding momentum. This will keep the ball rolling on the slope.
2. Using stochastic gradient descent rather than true gradient descent. Thus, weights are updated on each training data.
3. Training multiple networks with different initial weights using the same data and then selecting the best one out of trained networks.

In the next part, we will discuss about how to develop an ANN application and its application example.

Starting from October, I will be a consultant for a public International Standard High School (ISHS) in Pacitan i.e. SMAN 1 Pacitan. As you know, Pacitan is the city where  the incumbent president of Republic of Indonesia Susilo Bambang Yudhoyono (SBY) was born and raised during his childhood. Pacitan is also known for its beautiful caves in which there are hundreds of them all over Pacitan region. Thus, Pacitan is also called as ‘city of 1001 caves’. In addition, Pacitan  has many beaches which several of them are quite popular and also  beautiful. Back to the topic, my first visit to SMAN 1 Pacitan is to share my knowledge about how to teach math using English as the instructional language. Basically, there are three important things that I discussed with teachers of SMAN 1 Pacitan i.e. content and delivery, language, and computer.

In term of the content, I emphasized on its clarity, organization, and usage in real life. For example, introducing statistics, we should first go with the definition of statistics (WHAT) followed by the reasons we need statistic (WHY),  stages in statistics and detail in each stage (HOW). Also, the content has to be concise and the appropriate learning media has to be chosen. Learning media can be audio, video, conventional board, etc. Regarding the delivery, since we adopt Student Centered Learning (SCL) method, after the concept has been delivered, we should make students to learn by experience through case study, group discussion, presentation, and practicum. The objective is to make students grasping the insights related to the delivered concept. More valuable benefits can be listed as follows.

1. Making students active (cognitive, affective, psycomotoric)
2. Students able to enhance their soft skills e.g. teamwork; presentation skills; communication skills
3. Training the analytical thinking of the students

Next is the language. Since it is an International Standard High School, we have to use English as instructional language. Nonetheless, we can not be very strict in term of language. Thus, bilingual (English and bahasa Indonesia) is more suitable for students level 10 and 11 since most of the students do not have adequate skill in English. It will be able to avoid mis-perception and misinterpretation during the material delivery. Also, the transliteration and translation should be well managed, especially in introducing new terminologies.

Lastly, we should utilize IT applications in conjunction with the material that we have delivered if it is applicable/necessary. In my example e.g. statistics, we may use MS Excel to solve a given case study or problems. For instance, we can use MS Excel to create frequency table, present results using graph, calculate properties of averages (mean, median, mode), etc. By doing this, students will be familiar with computer and software.

During the discussion with the teachers, I withdrew some skills that they want to improve which mostly related to the utilization of IT applications in order to do the following.

1. creating attractive media e.g. flash, video, screen activity capture, etc
2. choosing suitable tools for practical experience for certain topics
3. finding reference in the Internet such as ebook

Moreover, some teachers wanted to be tutored in programming using PASCAL to support them in teaching programming to the students. This is one of the school program to promote their students in high school level programming olimpics. Now, it is my job to fulfill the teachers’ requests. Hopefully, I will be able to do the job with excellent results.

Here, I will discuss about neural network and its applications to improve business decision. Prior explaining about neural network, let me discuss about the idea behind the use of neural network to make the machine able to learn. Also, it is necessary to observe the elements and their basic functions that form neural network. Basically, artificial neural network is created to imitate the structure and work of neuron. Figure 1 exhibits neuron with its elements. The neuron is the cell in humans used to detect the outside environment, the internal environment of their own bodies, to formulate behavioural responses to those signals, and to control their bodies based on the chosen responses. Neurons are highly interconnected with each other forming a network. Hence, we call the interconnected neurons as neural network. Each neuron possesses axons and dendrites, finger-like projections enable the neuron to communicates with its neighbouring neurons by transmitting and receiving electrical and chemical signals.

Figure 1. Structure of Neuron

However, neural networks (NN) only represent a brain metaphor for information processing, not the exact structure and actual functions of the brain.

The earliest model of neural network is perceptron where there is no hidden layer introduced in this model (see Figure 2). For more detail regarding perceptron, please visit homepages.gold.ac.uk/nikolaev/311perc.htm.

Figure 2. Perceptron

From Figure 2, we can see that there are three important parts in Perceptron i.e. input layer, (a set of nodes), sigmoid function, and output (a node or set of nodes depends on the design of the network). Basically, the training in perceptron as well as in NN is to train/adjust the network weight towards given input and classification output  of the training data. In addition to perceptron, NN adds one more layer called the hidden layer. Figure 3 exhibits the structure of NN.

Figure 3. NN

As we can see in Figure 3, each node in each layer in perceptron and NN is connected with all nodes in the next layer. In designing an NN learning, there is no definitive rule that should be followed. Thus, it depends on the problem domain that we want to solve, the characteristics of the data, and the desired output. The most notable abilities of NN is its ability to solve complex non linear decision boundary very well.The common algorithm used in NN is back progagation algorithm (gradient descent). Other such as evoluationary algorithm (simulated annealing, EM) also can be employed in NN learning.

NN has been successfully applied to broad range of data-intensive applications, such as:

1. Industrial application such as process modeling and control, inventory management, targeted marketing (finding the set of demographics which have the highest response rate for a particular marketing campaign), and quality control.
2. Financial application such as portfolio management. For example, someone wants to plan his assets allocation in a portfolio in a way that maximises return and minimises risk. Another example is financial forecasting application to predict the future movement of security, stocks, bonds, etc.
3. Military application e.g. target recognition application which uses video and/or infrared image data to determine if an enemy target is present.
4. Medical application such as diagnosis application to assist doctors with their diagnosis by analysing the reported symptoms and/or image data such as MRIs or X-rays.
5. Banking application e.g. credit rating that automatically assign someone’s credit rating based on his/her financial condition. Other potential application is fraud detection to detect fraudulent credit card transactions and automatically decline the charge.
6. Linguistic/security application e.g. voice recognition to transcribe spoken words into ASCII text or to authenticate user to  access a system.
7. Information retrieval (IR) application e.g. intelligent searching by providing an Internet search engine that provides the most relevant content based on the users’ past behaviour.

In relation with business intelligence, NN is usually utilized for industrial, financial, banking, and IR applications. There are several database application products provide data mining algorithm using NN as one of optimization techniques for further business analysis and decision making e.g. MSSQL Server 2008 and Oracle BI. In addition, there is a data mining open source project that also employed NN i.e. WEKA software. In the next chapter, we will discuss about NN algorithm in detail and also software that employed NN for data mining.