GEMM-eMFIS (FRI/E) : a novel general episodic memory mechanism for fuzzy neural networks

One of the hardest challenges for machine learning models in finance, medicine, engineering, and science is to make real-time predictions during periods of sudden and large changes in the data from a steady state, known as transient behavior. An example is that a stock’s price can suddenly come crashing [54] upon events in the market such as its corporate actions, global recessions or breaking news, which would be of interest to both governments and private corporations. The idea is that it would be possible for a computationally intelligent system (CIS) to keep a memory of these transient events, learn the most relevant rules and reuse them when similar events occurs. This type of thinking in humans stems from the theory of episodic memory [8] which allows the storage and recall of similar events, in order to take appropriate action. Compared with other CIS’, Neuro-Fuzzy Systems (NFS) integrate Artificial Neural Networks (ANN) and Fuzzy Inference Systems (FIS) to possess the best of both worlds of learning complex representations in data while maintaining human interpretable rule sets [1]. They are known to perform well in the domain of Finance [5, 6] due to their ability to adaptively learn rules in real-time and have self-evolving structures, making them suitable for realizing an episodic memory mechanism. However, while recent NFS’ possess the ability to deal with real-time data and can self-evolve to deal with concept shifts, they still struggle during periods when the prices fluctuates wildly due to sudden changes in the data, for example shocks in the market from events such as recessions, news and corporate actions. Dealing with transient behaviour in the stock market proves to be challenging and many models do not have this capability. This paper proposes a novel general online episodic memory mechanism and demonstrates an instance of its integration into the Neuro-Fuzzy system architecture called Episodic Memory Based, evolving Mamdani Fuzzy Inference System with Fuzzy Rule Interpolation and Extrapolation (GEMM-eMFIS (FRI/E)). Inspired by established theories that govern human memory such as Episodic Memory [8], Multi-Store Model [3] and Autobiographical Memory [4], GEMM-eMFIS (FRI/E) learns from past events, by storing and retrieving them from an episodic memory cache during event-driven transient behaviour, boosting performance, while potentially reducing the number of rules needed. In addition, GEMM-eMFIS (FRI/E) [5, 6] also has several in-built mechanisms that enable to learn effectively from continuous stream of online data namely (1) Bienenstock Cooper Munro (BCM) learning theory to keep its rule base updated; (2) 2-Staged Incremental Clustering; (2-SIC) algorithm to determine cluster width; (3) Interpolation and Extrapolation of rules with to deal with concept shifts and drifts in the time-variant data; (4) Rule pruning and merging to keep the rule base compact. GEMM-eMFIS (FRI/E) is benchmarked against other NFS’ on various time-variant datasets such as stock index prices and rainfall runoff, and shows strong forecasting performances in those domains. The results are encouraging.Bachelor of Engineering (Computer Science