Home » Machine Learning » Reinforcement Machine Learning – Definition, Types, Algorithms, Examples and More

Reinforcement Machine Learning – Definition, Types, Algorithms, Examples and More

Table of Contents

Defining Reinforcement Machine Learning

Reinforcement machine learning is among the three main machine learning paradigms alongside supervised learning and unsupervised learning. As opposed to supervised learning where the goal is to find differences and similarities between data sets, Reinforcement learning strives to empower an agent to find the most suitable action and learn from its interactive environment. It uses rewards and punishments as signals for negative and positive results. By doing so the agent attempts to maximize the right moves as it minimizes the wrong ones.

Reinforcement Machine Learning Jargon

To keep you up to speed, here are important terminologies you should know when referring to reinforcement learning:

TerminologyDescription
AgentThe learner or decision maker that goes under the learning process. In this case, Artificial intelligence which is the algorithm is the agent.
ActionCumulative moves and agent can make.
EnvironmentThe space in which the agent operates and receives feedback. Consequently, the input is the agent’s current state while output is the rewards and the succeeding state.
StateA specific position in the environment or a moment in which an agent exists. The agent’s state can be current or future.
RewardAn agent receives from the environment a reward for every action it makes. The reward can be either negative or positive. 
PolicyBased on the current state, this is the approach the agent uses to decide the next action. It charts states action for the agent to choose the highest reward action.
ModelThis is how the agent views the environment. It charts the state action sets to the possibility distributions over states.
Value FunctionThis constitutes how suitable a state is for the agent. The states value means the long term reward an agent is due to receive starting from the current state to implementing specific policy.

Characteristics of Reinforcement Machine Learning

Important characteristics to note about reinforcement learning:

  • Feedback is not instant; it takes time to get an outcome.
  • Decision making is sequential.
  • Sub sequential data received determines the agent’s action.
  • Time takes an important part in reinforcement problems
  • No supervision, only a reward or an actual number

Types of Reinforcement Machine Learning

Reinforcement learning is broadly divided into two types: positive reinforcement and negative reinforcement.

Positive reinforcement machine learning

In positive reinforcement, algorithms receive a reward for a particular result. A good reward is added for every good result to cultivate more good results.

A good example is when training a dog. For it to learn faster it is rewarded after every task it performs, this way the dog is more motivated to do the masters command knowing it will get rewards for good performance. In this particular task, we are adding a reward for task completion thereby increasing the likelihood of the task being completed.

Positive reinforcement can have good results, especially in performance and change sustenance for the long term. The downside to this is that the system can have an overload that could impact the results negatively.

Negative reinforcement machine learning

Negative reinforcement is somehow different from positive reinforcement. In this case, a negative component is removed to improve performance.

An example to explain this: when a child does not do house chores at home, the parent might ground them as a way of punishment. This could mean no video games or no favourite snack. This way the child will make sure all the tasks given are done to get to avoid punishments.

From the above example, the algorithms in negative reinforcement will receive negative feedback. So to avoid a negative result, it will get rid of the processes that result in negative feedback. The advantage of this is that the attitude towards performing better will increase, assuring only positive results.

The disadvantage to this is that it will force you to meet the minimum requirements for the job to be complete.

Elements of Reinforcement Machine Learning

There are four important elements in reinforcement machine learning. They are reward signal, policy, model and value function.

ElementDescription
Reward SignalIn every stage, the agent receives a signal called a reward signal or just a reward. As we now know, rewards can be either positive or negative; this largely depends on the agent’s actions. If the reward signal is negative the agent can be forced to change the policy to justify its total reward.
PolicyA policy is a formula an agent uses to know the preceding action based on the current state. This element allows reinforcement learning to simply define an agent’s behaviour.
The role of a policy is to map out the perceived state of the actions taken by the states. It can either be stochastic or deterministic or a simple lookup table or a function.
ModelThe model copies the environmental behaviour. You can make conclusions about the environment and predict its behaviour. When a state and an action is provided, the next state of reward can be predicted using a model. This is good for planning since you can predict the future using a model. The approach used to solve reinforcement learning is called model-based reinforcement but if you decide to use a model in solving the reinforcement problem it is called a model-free reinforcement.
Value FractionThe main goal of a value function is to achieve more rewards by estimating values. The value function avails information about the favorability of specific actions and agents rewards, to simplify this it means value function determines which quality of a state is best for the agent to be in.

Reinforcement Machine Learning Algorithms

There are two commonly used algorithms in reinforcement learning. They can be classified as either:

  • Model-based algorithms – learns from current actions- or
  • Model-free algorithms – works on a trial and error basis. To further break it down we have algorithms that are on policy or off policy. The two algorithms are; State Action Reward State Action (SARSA) and Q-leading. They don’t require any model knowledge but only observed rewards from various experiments. Their only difference is their exploration strategies while their exploitation strategies are similar.

Q-learning uses an off-policy method where the action a* derived from a different policy is the learning point for the agent. SARSA on the other hand is using an on-policy method where the action a* derived from the current policy is the learning point for the agent.

The two algorithms lack generality but to achieve that more advanced algorithms are used like; Deep Q – networks(DQN) which have neutral networks that do estimations for Q – values and Deep Deterministic Policy Gradient(DDPG) which is model-free, off policy and actor-critic algorithm that learns policies in high dimensions continuously.

Reinforcement Machine Learning Examples

Reinforcement learning requires simulated and readily available data, especially in domains like game-play and robotics. Here are detailed examples of real-world applications of reinforced learning.

Applications in autonomous cars

Reinforcement learning has been applied when making an autonomous car, even though things like speed limits, driving zones. Collisions, breakage et cetera it is still a success thanks to the advanced algorithms behind reinforced learning.

Reinforcement learning has played a key role especially in areas like trajectory optimization, controller optimization, dynamic pathing and learning policies for highways.

Q-learning for instance has been used to achieve parking. Lane changing can be made easy by learning and avoiding collision while overtaking policy while overtaking can be achieved by learning automatic parking policies.

AWS Deep Racer, an autonomous racing car, is designed to test a reinforcement learning model to control the direction and throttle.

Applications in industry automation

Learning-based robotics are used in industry reinforcement in various ways, apart from being more efficient than humans, they can perform tasks that are hazardous to humans.

Deep-mind, Google’s artificial intelligent agent cools Google data centres and has proved to e more effective without human supervision. It feeds data of snapshots to deep neural networks every five minutes. It predicts how the various combinations will affect the future, identifies an option that will lead to minimized power usage and maintains safety measures put in place. Finally, it sends action points to the local control centre.

Applications in trading and finance

Time series models can be used to predict the future. Future stocks prices and sales can be foretold. Reinforced learning can advise whether to sell or hold. Market benchmark standards evaluate the model to monitor performance.
IBM has a very sophisticated reinforcement learning that can make financial trades. Based on the profit-loss report, it computes the reward function, this process brings consistency and better decision making.

Applications in healthcare

Patients can be treated based on policies learned through the reinforcement learning systems. A reinforcement system can find optimal policies using previous experiences and not previous mathematical models derived from biological systems.

In chronic diseases, automated care, critical care and other general domains reinforcement learning are categorized as dynamic treatment regimes (DTRs).

By factoring in the delayed treatment effects, Reinforcements learning has seen an improvement of long-term outcomes.

Applications in engineering

Horizon – Facebook’s open-source reinforcement learning platform- optimizes large scale production systems. Horizon is used to: deliver meaningful suggestions, optimize streaming of videos and personalize suggestions. Its workflows have simulated environments, data processing platforms and exporting and training models in production. Horizon’s capability ranges from feature normalization, deploying at scale, distributed learning, serving and handling high dimensional datasets and handling et cetera.

Applications in news recommendation

With reinforcements learning, tracking audience feedback based on reviews and behaviours could be fast. By following the customer preference closely, reinforcement learning can track the audience return behaviours.

The reinforcement learning system can have features like context features and reader features. Reader features interpret how an audience interacts with content while content features could mean freshness, relevance et cetera. A reward is then placed based on the user behaviour.

Instances Where Reinforcement Machine Learning is Futile

Reinforcement learning can be very helpful but in some circumstances, it can be rendered redundant. Here are a few conditions that you should not use reinforcement machine learning:

  • When there’s enough data to solve the Reinforcement learning problem with a supervised learning method.
  • When the action space is large. Remember that computing reinforcement learning is time-consuming and computing heavy.

Challenges of Reinforcement Machine Learning

When doing reinforcement learning you might encounter a few challenges like:

  • Realistic environments can be static
  • Speed of learning can be affected by different parameters
  • Realistic environments can bring out partial observability.
  • A lot of reinforcing leads to overloading of the state which in turn diminish results.

Conclusion

Significant research has been made on reinforcement learning and tremendous progress has been witnessed in the field and application in real life.

Reinforcement learning is by far one of the latest disruptions positively influencing different industries in areas of technology and has a huge potential to transform the world. This has been witnessed in Deepmind’s now popular AlphaGo streamed movies which not long ago was just an idea.

Thus, reinforcement learning has the potential to transform Artificial Intelligence for the better.

Topics in Machine Learning

Hits: 49