Aug 17, · A TensorForce-based Bitcoin trading bot (algo-trader).Uses deep reinforcement learning to automatically buy/sell/hold BTC based on price history. This project goes with Episode 26+ of Machine Learning allcryptocoins.de episodes are tutorial for this project; including an intro to Deep RL, hyperparameter decisions, etc. Algorithmic trading is full of data and calculations with the data. To deal with it, tensors (multidimensional data arrays) are ideal mathematical entities. And Tensorflow is the right software to . Tensorflow with Python, Keras, and Machine Learning to Trade Keras and TensorFlow + robot based on Tensorflow for cryptocurrency market Tensorflow: a system varying, Market efficiency With Tensorflow and for training reinforcement learning interval, Deep convolutional autoencoder trade robot based on Slight Pullback - Bitcoin for large.
Tensorflow bitcoin tradingGitHub - lefnire/tforce_btc_trader: TensorForce Bitcoin Trading Bot
Here we introduce the general workflow of TensorFlow Algorithms. This workflow can be follow as a template. This is usually the first step. Here you import libraries and modules as needed.
Also, load environment variables and configuration files. All of machine learning algorithms depend on data. So, we either generate data or use an outside source of data. Sometimes it is better to rely on generated data because we will want to test the expected outcome. Most times we will access market data sets for the given research. The raw dataset usually has faults which difficult the next steps.
In these steps, we proceed to clean data, manage missing data, define features and labels, encode the dependent variable and dataset time alignment when necessary. This step is useful when you need to separate data into training and test sets. We can also customize the way to divide the data. Sometimes we need to support data randomization; but, a certain type of data or model type needs the design of other split methods.
In general, the data is not in the correct dimension, structure or type expected by our TensorFlow trading algorithms. We have to transform the raw or provisional interim data before we can use them. Most algorithms also expect standardized normalized data and we will do this here as well. Tensorflow has built-in functions that can normalize the data for you. Some algorithms require normalization of the data before training a model. Other algorithms, on the other hand, perform their own data scale or normalization.
So, when choosing an automatic learning algorithm to use in a predictive model, be sure to review the algorithm data requirements before applying the normalization to the training data. Finally, in this step, we must have clear what will be the structure dimensions of the tensors that are involved in the input of data and in all calculations. Output: two datasets transformed training dataset and transformed test dataset. It may be, this step is accomplished several times given several pairs of train-test datasets i.
Algorithms usually have a set of parameters that we hold constant throughout the procedure i. It is a good practice to initialize these together so the user can easily find them.
TensorFlow will modify the variables during optimization to minimize a loss function. To accomplish this, we feed in data through placeholders. Placeholder simply allocates a block of memory for future use. By default, placeholder has an unconstrained shape, which allows us to feed tensors of different shapes in a session. We need to initialize variables and define size and type of placeholders so that TensorFlow knows what to expect. After we have the data and initialized variables and set placeholders, we have to define the model.
This is done by mean of the powerful concept of a computational graph. The graph nodes represent mathematical operations, while the graph edges represent the multidimensional data arrays tensors that flow between them. We tell Tensorflow what operations must be done on the variables and placeholders to get our model predictions. Most TensorFlow programs start with a dataflow graph construction phase. Operation node and tf. Tensor edge objects and add them to a tf. Graph instance.
After defining the model, we must be able to evaluate the output. THere we set the loss function. The loss function is very important a tells us how far off our predictions are from the actual values. There are several types of loss functions. Now that we have everything in place, we create an instance or our computational graph and feed in the data through the placeholders and let Tensorflow change the variables to predict our training data. TensorFlow provides a default graph that is an implicit argument to all API functions in the same context.
Here is one way to initialize the computational graph. Once we have built and trained the model, we should evaluate the model by looking at how well it does on new data known as test data. This is not a mandatory step but it is convenient.
The initial neural network is probably not the optimal one. So here we can tweak a bit in the parameters of the network to try to improve them.
Then train an evaluate again and again until meet the optimization condition. As result, we get the final selected network. Yeees, this is the climax of our work!. We want to predict as much as possible, It is also important to know how to make predictions on new, unseen, data. The readers can do this with all the models, once we have them trained. So, We could say that this is the goal of all our algorithmic trading efforts. Output: A prediction. You can have runs table in your history database if you want, one-and-the-same.
I have them separate because I want the history DB on localhost for performance reason it's a major perf difference, you'll see , and runs as a public hosted DB, which allows me to collect runs from separate AWS p3. Then, when you're ready for live mode, you'll want a live database which is real-time, constantly collecting exchange ticker data.
Again, these can all 3 be the same database if you want, I'm just doing it my way for performance. I have them broken out of the hypersearch since they're so different, they kinda deserve their own runs DB each - but if someone can consolidate them into the hypersearch framework, please do. In my own experience, in colleagues' experience, and in papers I've read here's one - we're all coming to the same conclusion.
We're not sure why Maybe LSTM can only go so far with time-series. Another possibility is that Deep Reinforcement Learning is most commonly researched, published, and open-sourced using CNNs. This because RL is super video-game centric, self-driving cars, all the vision stuff. So maybe the math behind these models lends better to CNNs? Who knows. The point is - experiment with both. Report back on Github your own findings. So how does CNN even make sense for time-series?
Well we construct an "image" of a time-slice, where the x-axis is time obviously , the y-axis height is nothing A change in TensorForce perhaps? TensorForce has all sorts of models you can play with. PPO is the second-most-state-of-the-art, so we're using that. DDPG I haven't put much thought into. Those are the Policy Gradient models.
We're not using those because they only support discrete actions, not continuous actions. Our agent has one discrete action buy sell hold , and one continuous action how much? Without that "how much" continuous flexibility, building an algo-trader would be You're likely familiar with grid search and random search when searching for optimial hyperparameters for machine learning models. Random search throws a dart at random hyper combos over and over, and you just kill it eventually and take the best.
Super naive - it works ok for other ML setups, but in RL hypers are the make-or-break; more than model selection. That's why we're using Bayesian Optimization BO. See gp. BO starts off like random search, since it doesn't have anything to work with; and over time it hones in on the best hyper combo using Bayesian inference.
Super meta - use ML to find the best hypers for your ML - but makes sense. Wait, why not use RL to find the best hypers? We could and I tried , but deep RL takes 10s of thousands of runs before it starts converging; and each run takes some 8hrs. BO converges much quicker. I've also implemented my own flavor of hypersearch via Gradient Boosting if you use --boost during training ; more for my own experimentation.
We're using gp. It uses scikit-learn's in-built GP functions. I found gp. But luckily I hear you can pretty safely use BO's defaults. If anyone wants to explore any of that territory, please indeed! GPL bit so we share our findings.
Community effort, right? Boats and tides. FYI, I haven't made a dime. Doubtful the project as-is will fly. It could benefit from add-ons, like some NLP fundamentals functionality.
But it's a start! Skip to content. TensorForce Bitcoin Trading Bot ocdevel. Go back. Launching Xcode If nothing happens, download Xcode and try again. Latest commit. Colab notebook. Git stats commits. Failed to load latest commit information. Aug 17, Upgrade to latest TensorForce. Remove scaling implicit in chang…. Feb 28, Support parallel GP. Nov 21, Jan 25, Aug 14, Remove custom net use tforce CNN netspec.
Roll stationary into time…. Aug 2, Jan 20, Update hypersearch. Aug 16, Misc updates. Feb 6, WIP tests. Jan 24, Feb 13, View code. Setup Python 3.