Leonid Peshkin's Research on Stigmergic Learning Policies with External Memory

Title: Leonid Peshkin's Research on Stigmergic Learning Policies with External Memory

Abstract: Leonid Peshkin, a computer scientist at Brown University, proposed a novel approach to reinforcement learning in partially observable environments. His method, called stigmergic learning, involves the use of external memory devices. The agent's actions can include setting and clearing bits in this memory, and the memory is included as part of the input to the agent. Peshkin explored two algorithms: SARSA (λ), which has had empirical success in partially observable domains, and VAPS, a new algorithm by Baird and Moore, with convergence guarantees in partially observable domains. He compared the performance of these two algorithms on benchmark problems.

Research Question: How effective are stigmergic learning policies with external memory in partially observable environments?

Methodology: Peshkin's research involved developing two algorithms: SARSA (λ) and VAPS. SARSA (λ) is a reinforcement learning algorithm that can perform well in partially observable domains. VAPS, on the other hand, is a new algorithm with convergence guarantees in partially observable domains. Both algorithms were tested on benchmark problems to evaluate their performance.

Results: Peshkin compared the performance of SARSA (λ) and VAPS on several benchmark problems. He found that both algorithms performed well, indicating that stigmergic learning policies with external memory are effective in partially observable environments.

Implications: Peshkin's research suggests that stigmergic learning policies with external memory can be an effective approach to reinforcement learning in partially observable environments. This could have significant implications for the field of artificial intelligence, as it may provide a new way to design more efficient and effective learning algorithms for complex, real-world problems.

Link to Article: https://arxiv.org/abs/0103003v1 Authors: arXiv ID: 0103003v1