Backpropagation Through Time: What It Does and How to Do It PAUL J. WERBOS Backpropagation is now the most widely used tool in the field of artificial neural networks. At the core of backpropagation is a

Source URL: www.cs.cmu.edu

Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies Sepp Hochreiter Fakult¨at f¨ ur Informatik

Source URL: www.bioinf.jku.at

• Artificial neural networks
• Applied mathematics
• Cybernetics
• Backpropagation through time
• Recurrent neural network
• Exponential growth
• Backpropagation
• Mathematics

International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems cfWorld Scientic Publishing Company THE VANISHING GRADIENT PROBLEM DURING LEARNING RECURRENT NEURAL NETS AND PROBLEM SOLUTIONS Sepp Hochreiter

Source URL: www.bioinf.jku.at

• Artificial neural networks
• Recurrent neural network
• Long short-term memory
• Vanishing gradient problem
• Sepp Hochreiter
• Backpropagation through time
• Backpropagation
• Artificial intelligence
• Deep learning
• Speech recognition

LONG SHORT-TERM MEMORY Neural Computation 9(8):1735{1780, 1997 Sepp Hochreiter Fakultat fur Informatik

Source URL: www.bioinf.jku.at

• Artificial neural networks
• Computational neuroscience
• Digital electronics
• Digital systems
• Electronic engineering
• Long short-term memory
• Sepp Hochreiter
• Backpropagation through time
• Backpropagation
• Flip-flop
• Memory cell
• Artificial intelligence

Recurrent Neural Net Learning and Vanishing Gradient International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6(2):107{116, 1998 Sepp Hochreiter Institut fur Informatik

Source URL: www.bioinf.jku.at

• Artificial neural networks
• Recurrent neural network
• Long short-term memory
• Vanishing gradient problem
• Sepp Hochreiter
• Backpropagation through time
• Backpropagation
• Artificial intelligence
• Artificial neuron
• Time delay neural network
• Deep learning
• Types of artificial neural networks

Evolving memory cell structures for sequence learning Justin Bayer, Daan Wierstra, Julian Togelius and J¨ urgen Schmidhuber IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland {justin, daan, julian, juergen}@idsia.ch

Source URL: people.idsia.ch

• Artificial neural networks
• Machine learning
• Applied mathematics
• Cybernetics
• Long short-term memory
• Recurrent neural network
• Artificial intelligence
• Handwriting recognition
• Feedforward neural network
• Backpropagation through time
• Backpropagation
• Brain

Workshop track - ICLRF EED -F ORWARD N ETWORKS WITH ATTENTION C AN S OLVE S OME L ONG -T ERM M EMORY P ROBLEMS Daniel P. W. Ellis LabROSA, Columbia University

Source URL: colinraffel.com

• Artificial neural networks
• Computational neuroscience
• Artificial intelligence
• Computational statistics
• Deep learning
• Recurrent neural network
• Speech recognition
• Sepp Hochreiter
• Long short-term memory
• Yoshua Bengio
• Backpropagation through time
• Memory

Sequence-to-Sequence Neural Net Models for Grapheme-to-Phoneme Conversion Kaisheng Yao, Geoffrey Zweig Microsoft Research {kaisheny, gzweig}@microsoft.com

Source URL: research.microsoft.com

• Artificial neural networks
• Machine learning
• Applied mathematics
• Cybernetics
• Long short-term memory
• Recurrent neural network
• Speech recognition
• Bidirectional recurrent neural networks
• Artificial intelligence
• Language model
• NETtalk
• Backpropagation through time

LONG SHORT-TERM MEMORY Technical Report FKI, Version 3.0 Sepp Hochreiter Fakultat fur Informatik Technische Universitat Munchen

Source URL: www.bioinf.jku.at

• Artificial neural networks
• Long short-term memory
• Sepp Hochreiter
• Backpropagation through time
• Backpropagation
• Artificial neuron
• Artificial intelligence
• Control theory
• Memory cell
• Recurrent neural network
• Deep learning

BackPropagation Through Time Jiang GuoAbstract This report provides detailed description and necessary derivations for the BackPropagation Through Time (BPTT) algorithm. BPTT is often

Source URL: ir.hit.edu.cn