This article pits Equities Lab and Gurufocus against each other to determine which one deserves the individual investor’s loyalty, but only once the criteria for a “good” backtesting tool are specified.
Investopedia.com states that backtesting is “the process of testing a trading strategy on relevant historical data to ensure its viability before the trader risks any actual capital. A trader can simulate the trading strategy over an appropriate period of time and analyze the results for the levels of profitability and risk.”
Breaking this down, it’s apparent that backtesting software must be able to screen a selected universe of stocks to find ones that match a set of criteria. The software also needs to be able to do this in the past (or using historical data), and it would be nice if the user could run the screen at any point in the past. Beyond that, investors typically use a blend of fundamental and technical metrics, liquidity constraints, real-world portfolio size limits, and other details. Some investors include the limitation of using stop loss orders and others like making up their own factors. Ideally, a good backtesting program would be able to handle all these tasks easily.
Several other factors conspire to narrow the field (to prevent this article from becoming a book). Individual investors won’t spend much more than about $250 per month. This excludes Bloomberg, FactSet, Eikon, Palantir, Zack’s Research Wizard, and other capable packages. Programs that don’t have easy-to-use automated backtesting capabilities are excluded (Some examples are Gurufocus, Stock Investor Pro, and OldSchoolValue), as are programs that only do technical analysis (Examples are AmiBroker, TradeStation, and Weath-Lab). Finally, programs that require the user to write programs (Quantopian, QuantConnect, etc) are excluded, so this article stays an article and not a book. From what’s left, I chose based on popularity, notability, features, and power.
The table below summarizes each package’s monthly cost, rounded to the nearest dollar.
|
Cost |
Equities Lab |
Gurufocus |
|
Monthly cost for the most powerful version |
$100/month for Institutional Membership |
$100/month for PremiumPlus membership |
|
Notes and extras |
No annual pricing on the website. |
Only option to purchase annually Can purchase upgrades to include Asian, European, Canadian, UK, Oceania, Latin American, African, and Indian securities. |
How I wrote this article
Is it a coincidence that Equities Lab is first on the list? For that, you’ll have to blame the English language and its wacky ordering of the alphabet. But, jokes aside, I work at Equities Lab, meaning I had to go above and beyond to avoid bias. Two opposing sorts of biases have to be opposed: the bias to say my product was best and the inside view, with its intimate knowledge of each and every one of Equities Lab’s warts.
I worked against these biases by being detail-oriented: I broke each package down into five categories, with a double handful of tests for each category, and detailed why each package got the result it did on each subtest. The tables below are exhaustive, providing a comprehensive torture test for each software package. Only after these were complete and other special abilities were identified for each package did I try to create any summation or overall grade. And though I believe I was as unbiased as possible in this review, the bias of knowing how to use the software could have introduced some issues. If you see something about Gurufocus that you don’t believe is accurate, feel free to comment, and we will look into changing the grade.
Comparing the Tools
For easy comparison, each product is graded on the features an investor would want. Comments explain why each product got the grade it did on the categories.
The Grading System
The grades in this table and all the ones below are as follows:
|
A+ |
Amazing. Stands out well beyond its peers, making it the clear leader. May have flaws, but the overall experience is truly exceptional. |
|
A |
Very Good. Goes above and beyond the ordinary, giving a better experience. |
|
B |
Good. It is nice to use, or contributes to the value the package provides. No program can be good everywhere, but it is a goal to aspire to. |
|
C |
Does what it is supposed to, give or take. It does the job well enough, without causing annoyance, or requiring too many workarounds. |
|
D |
Worth avoiding. It mostly does what it is supposed to, after a fashion… with some caveats. |
|
X |
Not part of the program. It is simply missing, and the program makes no claim to offer it. |
|
F |
Unusable. It looks like it works, but doesn’t actually do the job it is supposed to – at all. There are issues that can’t be worked around. F- |
|
F- |
Deceptive. Using this functionality is dangerous, and may cause users to make bad investments. |
Not all grades are created equal – packages getting a low grade on critical functionality will get a lower overall score. Each category that matters is explained below, with a table grading the packages, along with a “overall” grade for the category.
Power
This refers to the ability of the program to model what the user is trying to model. Can it put its finger on what the user wants, or only make a crude approximation? There’s a clear tradeoff here: a C programmer can make a computer do exactly what she wants it to, given enough time, while a one-button interface gives lots of “power” given its non-existent complexity.
This table measures the amount of power available in a “reasonable” amount of effort or time. Easy things should be easy, hard things should be moderate, and the nearly impossible should be doable. The power measured here is also what can be done within the program – rather than by running a bunch of screens and using Excel to compose the results. Programs with useful functions that can compose those useful functions well will end up with the edge here. Users need to be able to compare quantities (PE > 11), rank quantities (PE in the top 50%), compare quantities across time (PE larger than last year) and compose these operations together. They also need to be able to create and manage moderately complicated formulas (e.g. Piotroski score).
|
Power |
Equities Lab |
Gurufocus |
|
Screen in the past |
A Single Click |
A Single click |
|
Simple (PE > 11) |
A Just type it in, or find it in tools menu |
A Just type it in, or find it in Valuation menu |
|
Changes of quantities (PE went down by 20% in the last year) |
A+ Just type it in. Any field can be lagged any amount, easily |
C The have lagged fields, but cannot customize them |
|
Ranking stocks (PE in the bottom 30% of the market) |
A+ Just type it in, any field or equation can be ranked easily Multiple ranks possible within a screen. Can rank with sector, industry, or custom group, and can filter. Nesting is easy. |
X No rank operators |
|
Simple formulas (Piotroski > 7 and Low Price to Book) |
A+ Piotroski exists as an importable, customizable formula. Piotroski can also be built from scratch, easily, saved and reused. |
A Piotroski exists as a standard formula. Cannot customize it. Could build it from scratch |
|
Complex expressions (Low Stddev of change PE over one month) |
A+ Just nest the expressions. Functions operate on time series, and across sections of the market, and all can be nested. Some can be slow (e.g. a moving average of a rank of PE across the market). |
X No expression to nest |
|
Cross market computation (total earnings of the largest stocks) |
A+ Cross sectional expressions are just like other expressions, and are usable everywhere. |
X No cross sections |
|
Expected sub-categories (include only airlines, the S&P 500, etc) |
D Sectors and industries included, but no indexes or exchanges. Usable proxies for S&P, Russell aren’t exact. |
B+ Includes sectors, industries and countries and some Indexes. |
|
Overall |
A+ Extremely powerful tool. |
B Fairly powerful tool. Lacks some customizability |
Data
Investment programs live on their data and how well it supports backtesting and screening in the past. Investors need data that covers at least one major market cycle, though two is better. They also need fundamental data (earnings, revenue, assets, liabilities, etc.), as well as technical analysis (the ability to play with patterns in the price data). Macroeconomic data (US GDP, inflation, etc.) is very useful, though not critical. Lastly, they may find earnings estimates, ownership information, and foreign data sets useful.
|
Data |
Equities Lab |
Gurufocus |
|
Length of history |
A 22 years |
C 12 years |
|
Fundamental data |
A+ Comprehensive set of 600+ fundamental metrics, not including ratios |
B Lot’s of raw data, fundamental data, and metrics |
|
Technical data |
A Moving averages, RSI,MACD, triangles, wedges, a few candlesticks, and the ability to add your own with lagging and sequence operators |
D Includes basic technical data |
|
Ownership data |
X No ownership data |
A+ Has data and can dig into ownership on a deeper level |
|
Earnings Estimates |
X No earnings data |
B Very Small amount of earnings data. |
|
Macroeconomic data |
A Seamless integration with quandl allows usage of all quandl data sets including paid ones (if you have subscribed). |
X No Macro Data |
|
Alternative data sets |
B Quandl gives access to large body of data. No foreign stocks. ETFs included, but cannot screen ETF’s. |
A 8 different American exchanges, different countries. All screenable, no ETF’s |
|
Overall Data |
A A focus on the core: Amazing fundamentals (600+ fields) and solid technical analysis capabilities make the data solid, useful and powerful. |
A A backtester needs to either be able to run to the current day, or at least screen what it backtests, to be useful. FinViz does neither. |
Usability
Usability is not the same as simplicity or beauty. It also refers to how easily the user can use the program to complete something, assuming it can be done. Aesthetics, ease of getting started, ease of navigation, and speed all matter. However, users need to read and modify what they (or others) have written, so readability and ease of editing matter. Lastly, the pre-built content included enhanced learnability and usability.
|
Usability |
Equities Lab |
Gurufocus |
|
Aesthetics |
B Good use of colors and shapes. Utilizes the design to help teach the user. |
C Very dense and difficult to follow |
|
Deployment |
B Windows, Mac, or Linux only. Does not require admin rights to install. |
A+ Web, very good mobile |
|
Starting a new screen |
B Suffers from the blank screen effect, but clicking on the + generates new lines, joined together with ‘and’. Has sensible defaults that are easy to change. |
A Creating a new screen is as simple as choosing various choices. |
|
Tooltips, searchability, and autocomplete |
B Autocomplete is effective, but does not pop up help while completing. Users need to use the tools pane for that. |
B No searching as everything is placed in the “All” tab. To read more about the item you simply click on it. Can quickly become difficult. |
|
Expression readability |
A+ Tooltips explain functions and fields, so reading unfamiliar expressions is easy. Easy use of variables and formulas help chunking. |
A All expression are in drop down form and never leave that form. Easy to read each line. |
|
Editing existing content |
A+ Drag and drop, extract variable or formula, all easy wrapping of terms all work together to make editing joyful. |
A Editing doesn’t go that deep, but, for the features editing is as simple as typing or selecting from a drop down. |
|
Cut and paste |
A+ Any term can be cut and pasted, even into notepad. Multiple cut buffers are a nice touch. |
X No copy and paste |
|
Navigation through screens |
A Fast navigation, searchable tabs, ctrl-Click to go into an item, and a breadcrumbs trail make navigation easy. |
B Fairly easy to navigate. Made a bit more difficult when on the “All” tab as the screen becomes cluttered and hard to navigate |
|
Preloaded content library |
A+ Large, searchable content library. 500+ screens (many of them segments, like “Low P/E”), and 400+ formulas give the user plenty to work with. Piotroski, Beneish, Altman, Montier, etc are all done out, so users can modify them. 500+ formulas to use as building blocks. |
D Only 15 prebuilt screens that can be backtested. |
|
Content reuse |
A+ Both variables and formulas help ensure content reuse. Formulas can have descriptions. These allow users to maintain criteria they use in all their screens (such as no penny stocks) |
X No way to create content to reuse. |
|
Handling user error |
A+ Much user error is prevented by the editor, with reasonable error messages. Multiple level undo and a restore autosave make it difficult to lose work by mistake. |
B Much user error is prevented by the use of a dropdown menu only system. No undo. |
|
Speed |
A+ UI is amazingly fast, and handles massive data well (100,000 trades in a table). Some queries are slow. |
C One of this systems biggest drawbacks is the speed. It took 5 minutes to run a simple screen of PE > 11 with only 50 positions. Had to restart three times to get it to work |
|
Overall Usability |
A An actual joy to use (after the first few minutes of bafflement). The preloaded content inspires the user to new heights. Can be slow sometimes. |
B Eminently approachable and usable – if not useful. Its simple and intuitive, Everything is just a bit jumbled up and works a bit slow. |
Configurability
Configurability lets users bend the software to their will rather than vice versa. It is essential for people building and tweaking an investment system to be able to find, understand and twiddle the knobs that may drive performance. Being able to control when backtests “rebalance” or replace all their holdings with new ones helps to ensure the system being used is robust. A strategy that works when rebalanced monthly but fails weekly or quarterly is probably a disaster. Similarly, while swearing by a stop-loss regime is good, being able to test several is better. Being able to get useful data about the results lets users be more confident in the results.
|
Configurability |
Equities Lab |
Gurufocus |
|
Rebalance periods |
A+ Fully configurable rebalance, though one must rebalance all stocks, or none. |
C Limited to no rebalance 3-month, 6-month, and 12-month. |
|
Buy/Sell without rebalancing |
C Can sell without rebalancing, but not buy. Sales go to cash until the next rebalance. |
X Cannot buy or sell outside of rebalancing in backtests |
|
Stop losses/Stop gains |
A Custom stop losses/gains possible. Stop losses and gains can be adjusted based on conditions. Trailing stops possible. |
B Custom stop loss, but no stop gain. |
|
Minimum/maximum holding periods |
A+ Minimum and maximum holding periods can vary per stock and change over time. |
X No time based stops |
|
Data about the backtest |
A+ Can get a zoo of statistics over time, including holdings, and even use these in the backtest (Stop trading if a threshold is met, etc). Basic plotting is easy, more complicated uses are less obvious. |
C Not a large amount of data for the backtest. Most of the data is presented around the screener |
|
Backtests selling based on other backtests. |
A+ The results of backtests are just another expression to be used. |
B Can kind of do it by setting an exit when screener. |
|
Overall |
A+ Virtually everything can be configured – arbitrarily. Options to control leverage and short selling are hard to find and use. |
B Configurable, assuming you don’t want to use all of the matches as positions or want to rebalance on anything more often than quarterly. |
|
Analytics |
Equities Lab |
Gurufocus |
|
Tables |
A All results in one page, searchable, sortable by multiple categories, exportable |
C Sort columns, only see 20 at a time |
|
Backtest performance chart |
A Configurable benchmarks, other backtests, and data on chart, which is clickable, zoomable and more |
D Only 4 lines – can’t change. Given some statistics on the backtest page. Slightly interactive. |
|
Plot a single value or expression (What is this company’s PE?) |
A+ Plots ad-hoc plots and panels, fields and expressions, in screens, backtests and charts. |
X No way to customize values plotted |
|
Configurable data display |
A+ All data within all pages is completely customizable. Can build tear sheets to supplement those in the system. |
X No data customization within backtests or screens |
|
Heatmaps |
A Heat maps used in most places throughout the software. All maps allow the adjustment of order, size, and what is considered a good or bad return. |
F Has heatmaps in single place in the site for valuation. Breaks when trying to map a screen |
|
Single Stock Display |
A Interactive charting with custom plotting, customized plotted panels, price history, Trading models, links to news and filings. |
A+ Interactive charting with custom plotting. Tons of historic information and filing information |
|
Overall |
A+ Backtest by time, positions breakdown, scatter charts, and good reports let users understand the results clearly. The ability to add plots to the backtests is very useful. |
C They lack some serious features in their backtesting and analysis. They do make up for it a bit in their individual stock charts. |
Analytics
A system that takes all the users’ input and returns a single number (the system returned 623%) would be useless. Systems at least need to show the performance curve, so users can see when and how the system did well. Systems that can give the users more data and make it understandable get the edge here. Good analytics also make it easy for users to express themselves and make them more confident that what they did in the past made sense.
Final Assessment
The final assessment pulls all the data from the tables above together into an opinion of how useful the package is for backtesting. Power, usability, configurability, and data all matter, but a package with a clear limitation that makes backtesting impossible or useless will get a low grade regardless of its other capabilities. An article that compared screening, charts, or performance on some other task would give these packages a higher grade.
|
Overall |
Equities Lab |
Gurufocus |
|
Power |
A+ Equities Lab is able to handle even very complex terms (e.g. Quality Minus Junk, Olhson’s O score, etc) |
B Gurufocus is incredibly powerful from a screening perspective. There are a lot of options that work fairly easily. |
|
Data |
A A focus on the core: Amazing fundamentals (600+ fields) and solid technical analysis capabilities make the data solid, useful and powerful. |
A Gurufocus has a ton of fundamental data, and their backtest/screener runs pretty far into the past to 2006. |
|
Usability |
A+ An actual joy to use (after the first few minutes of bafflement). The preloaded content inspires the user to new heights. Can be slow sometimes. |
B Eminently approachable and usable – if not useful. Its simple screener is intuitive, while their backtester is less so. |
|
Configurability |
A Virtually everything can be configured – arbitrarily. Options to control leverage and short selling are hard to find and use. |
B Configurable, assuming you don’t want to use all of the matches as positions or want to rebalance on anything more often than quarterly. |
|
Analytics |
A+ Backtest by time, positions breakdown, scatter charts, and good reports let users understand the results clearly. The ability to add plots to the backtests is very useful. |
C The lack of any (good) visuals beyond the performance chart make users of Gurufocus sad. |
|
Overall |
A+ Equities Lab validates and creates investable strategies more effectively than any other contender. |
B Gurufocus is a pretty good screener – especially for fundamentals. Their backtester leaves a bit to be desired, but it looks like their main focus is their screener. |
Narratives
For variety, the narratives go in ascending order (as is done in most awards ceremonies). Let the narratives begin!
Equities Lab (A+ as the whole is greater than the sum of its parts)
Equities Lab is quite a contrast from the other packages tested here. It presents users with a blank slate to start with and adds structure as needed to express what the user is inputting. This flexibility allows users to handle the sorts of systems that are featured in finance papers.
The analytics are as impressive as the fundamental analysis – the data can be mapped, sorted, exported, scatter charted, and just plain plotted. Not only is the user interface both pretty and efficient, supporting drag and drop, cut and paste, undo, and more.
That said, Equities Lab isn’t perfect either. It has no institutional, insider ownership data, and no earnings estimates. More irritatingly, it doesn’t have historical index membership, making screens like “Find the largest 50% of the S&P 500 by total assets” impossible. Yes, one can fake it by using the same rules Standards and Poors do, but it’s not the same.
Gurufocus (B it’s a really good tool for fundamental investors)
The fundamental analysis capabilities of Gurufocus are fantastic, but the analytical side of the system is a bit lacking. The user interface is fairly efficient – though a bit slow at times. You can get a handle on the software after watching an hour of tutorial videos on Youtube.
That said, Equities Lab isn’t perfect either. Gurufocus beats Equities Lab in a few areas, specifically institutional ownership, insider ownership data, and earnings estimates. Though it costs more money, you can add international data sets and segregate those sets.
Conclusion
Getting the right backtesting system is important, because of the sheer time you’ll put into your chosen product. Imperfections aside, Equities Lab and Gurufocus both are up to the task. Equities Lab is substantially more powerful, and easier to use, and that gives it the edge here. Both products got easier to use with time, but Equities Lab could do a variety of things that Gurufocus couldn’t. So, for fundamental and technical investors, Equities Lab seems the clear winner.