LSTM-ARIMA with Attention and Multiplicative Decomposition for Sophisticated Stock Forecasting.

A simple but complete full-attention transformer with a set of promising experimental features from various papers

心律失常检测/ECG信号分类

Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD

EQTransformer, a python package for earthquake signal detection and phase picking using AI.

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Text Summarization Modeling with three different Attention Types

Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)

Contrastive-LSH Embedding and Tokenization Technique for Multivariate Time Series Classification

Attention-guided Feature Distillation for Semantic Segmentation

A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

Corpus2GPT: A project enabling users to train their own GPT models on diverse datasets, including local languages and various corpus types, using Keras and compatible with TensorFlow, PyTorch, or JAX backends for subsequent storage or sharing.

Pre-training a Transformer from scratch.

A Jax-based library for designing and training transformer models from scratch.

VCR-Graphormer: A Mini-batch Graph Transformer via Virtual Connections, ICLR 2024

Implementation of Infini-Transformer in Pytorch

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

This repository contains the code for Deep Residual Feature Extraction and Spatial-Frequency Attention based Image Denoiser.

In this project, I developed a Machine Translation Engine for Vietnamese using the Transformer Architecture. My focus was on making the implementation clear and accessible, starting from the basics of the model's architecture and training process.