Title: ARE MINING POOLS BECOMING A PROBLEM?

Author: Bitcoin Mechanic
Compiled by: Luccy, BlockBeats

Editor's Note:

Mining is one of the solid and elegant designs conceived by Satoshi Nakamoto, and it is also one of the most striking aspects of Bitcoin. However, mining is not just about performing hash calculations.

Bitcoin advocate and educator Bitcoin Mechanic delves into the various issues and challenges present in Bitcoin mining, with a particular focus on the dynamic relationship between mining pools and miners. He proposes solutions to existing problems, aiming to enhance miners' autonomy and the overall trend towards decentralization.

The discussion on the StratumV2 protocol and the future development of the mining ecosystem is filled with unique insights. Additionally, the article highlights the latest developments in ASIC manufacturing and mining pool infrastructure, providing new possibilities for decentralization in the mining field. Through a critical analysis of existing mining models and a look at potential improvements for the future.

Bitcoin miners provide an extremely important service to the entire ecosystem. As part of ensuring network security, they receive rewards from the network, which is one of the solid and elegant designs conceived by Satoshi Nakamoto and one of the most striking aspects of Bitcoin.

However, more and more people seem to overlook that mining is not just about performing hash calculations.

Participants in the entire process must run a node to obtain the latest state of the blockchain and then start building a new block. This includes verifying the validity of the previous block, discovering unconfirmed transactions, and generally selecting the most profitable transactions among them. In the generated transactions, miners pay themselves, build multiple Merkle trees for these transactions, and finally perform hash calculations to actually solve the block. The transactions in the block template will constantly change as new transactions are broadcast to the network. When others find a new block, miners must switch to building on it and dump all transactions already in the blockchain to fill a new template.

Fork Activation

As you can see, performing hash calculations to actually solve a block is just part of this process. Bitcoin mining ASICs can only perform hash calculations. In the current environment, all other aspects of mining are typically delegated to mining pools. This leads to some confusion. For example, in any discussion about activating soft forks through version bit flips within block templates, people always mention that this process is a MASF, or "Miner Activated Soft Fork," and someone will always step up to say that this is the mining pool's full responsibility, and that mining pools are not miners. They may also point out that miners are ultimately still responsible because if they want to upgrade but their mining pool does not, they can simply switch pools. For clarity, in the remainder of this article, I will refer only to those who only participate in hash calculations and leave all other aspects of mining to the pools as "hashers."

Returning to soft forks, in the current environment, over 99% of blocks are built by the same organization, so it may be more accurate to call these "pool activated soft forks," although no one currently refers to them this way, leading to a dangerous illusion: that mining can be considered decentralized simply because of the distribution of hash power. When all hash power is confined to a few mining pools, this claim becomes implausible, and the contents of the Bitcoin blockchain will ultimately exclude anything deemed unacceptable by these few entities, along with a host of other issues.

By merely hashing blocks built by pools, Bitcoin miners largely relinquish a key component of their role. This is not only possible but also the smoothest path, indicating that we have a systemic problem.

Mining Pools and the Block Space Market

The impact of merely performing hash calculations and letting pools handle everything else goes far beyond soft fork activation. For example, miners currently have no idea what the solved block will look like, meaning they work under the blind trust that the block will only contain the expected transactions. But in blocks like these, you will see a blatant violation of that trust, as evidenced by the infamous block that sparked the "ordinal" craze. Note that the miners working on this block actually only enjoyed about $200 in Bitcoin transaction fees, while the average transaction fee for the blocks on either side was around $5,000 in Bitcoin.

Block space has value, which is part of why Bitcoin operates in the long term. But in a world where only a few participants can make their constructed templates ultimately enter the blockchain, these entities have almost exclusive rights to sell this space and reap excessive rewards. Are they responsible, or even obligated, to be transparent with their miners about what they are doing? Of course, the intent in this case is to catch everyone off guard. In the future, will they forward the payments they receive from selling out-of-band block space to their hashers?

In short, while the incentives of mining pools and their hashers are generally aligned towards maximizing profits, pools have the potential to sell block space for something other than regular Bitcoin transactions, while miners' income is more limited unless pools choose to be transparent and agree to share revenue. Even if they do, verification requires permission from the pool, rather than verifying funds earned from subsidies and transaction fees (which is also tricky when using FPPS pools, which will be elaborated on later).

The further impact of pools as builders of centralized Bitcoin block templates stems from a more fundamental fact - at a more basic level, there are twelve "super nodes" with their own "super mempools."

This leads to direct dealings with mining pools, completely ignoring the mempool. Some argue that the mempool is destined to fail anyway. The current centralized state of template building only accelerates this process, but in any case, it is undesirable, and making such an assumption in a world where truly decentralized template building is considered feasible may be overly pessimistic. Then, if those purchasing block space wish to incorporate it into the blockchain within the same timeframe, out-of-band payments must be passed to a larger crowd. This could be more transparent and similar to the current way of working. In contrast, "super nodes" are expected to be broken down into smaller parts, thus no longer able to provide the same guarantees.

To deviate from this aspect of mining, let’s shift our focus to how current payment methods are handled.

Mining Pool Payment Models

Almost all mining pools pay their hashers through FPPS (Full Pay Per Share) or similar methods. The only exception is ViaBTC, which offers PPLNS (Pay Per Last N Shares) in addition to FPPS. Antpool also offers PPLNS, but hashers must forfeit all transaction fee income, which illustrates the point I am about to clarify. Essentially, in a world focused on transaction fee income rather than subsidies, FPPS is not a well-functioning model. It is worth mentioning that the Braiins pool (formerly Slushpool) uses a system called "score," which is actually very similar to PPLNS.

Why such a strong preference for FPPS? From the hasher's perspective, they get paid regardless of what happens on the blockchain. This aligns with the purpose of pool mining, providing more consistent income. FPPS offers more consistent payments because pools pay based on expected income and settle independently of the blockchain.

This makes life very easy for miners who want to minimize issues caused by cash flow interruptions, but of course, there are downsides - significant downsides that I hope to emphasize here.

FPPS first requires pools to act as custodians of all newly mined Bitcoins. These Bitcoins cannot be forwarded to miners for at least 100 blocks after mining, meaning newly mined Bitcoins are unusable until then, and in reality, the mined Bitcoins have no relation to what miners ultimately receive when withdrawing from the pool. The risks of third-party custody should be apparent to almost everyone reading this, so I will skip over that and continue to introduce other issues with FPPS.

The next concern arises from a more general issue: FPPS pools are an important intermediary between hashers and the network itself. We have established that hashers cannot know in advance what the block they are working on will ultimately look like until it is solved. FPPS means they do not even care whether a block is found; that is the pool's problem. Ignoring the increased predictability of payments (which will never happen if the pool decides to deceive the hashers), we must acknowledge the trade-offs involved.

Miners paid directly by Bitcoin itself - which is possible in alternatives like PPLNS or, of course, solo mining - can expect to receive the full rewards, including transaction fees. An FPPS pool can only calculate this retrospectively because it is impossible to predict how much the fees will be when determining what each hasher actually receives per share. Pools cannot simply assume that fees will be some value greater than 0 and account for it in miners' income while mining, because if the fees are below that value, they will simply pay miners out of their own pockets. They must regularly allocate fees and attribute them to the miners actually held in the pool.

From the hasher's perspective, there is a need for complete trust in the pool, because without the pool's full transparency and cooperation, verification is nearly impossible. As mentioned earlier, this was not a major issue before, as most mining income came from subsidies, with transaction fees being just a few scattered satoshis among them, but this is not, and cannot be, the future of Bitcoin mining as it grows. In the future, miners will primarily earn income through transaction fees, and when using pools, these fees are harder to predict and monitor than subsidies.

Compared to payment schemes like PPLNS, hashers accept greater variability (the pool's luck also becomes the hasher's luck), and we see that the mining ecosystem is extremely inclined to prioritize the consistency of payments over the ability to verify what is received. More distorted is that some hashers actually prefer it this way - hoping to present themselves to government agencies as a "hash service" completely detached from Bitcoin - some even take pride in this. This is because FPPS is so far removed from the ideal miner/pool dynamic that it becomes difficult to describe what hashers are actually doing again, even as "Bitcoin mining."

In reality, FPPS pools are large independent miners paying hashers to solve their blocks. Afterward, they have an internal and opaque process through which they determine the amount paid to hashers. To illustrate this, hashers could even (in some not-so-hard-to-imagine scenarios) be paid in something other than Bitcoin.

Why not? If you do not care whether any blocks are found, let alone what they look like before they are built, why not just get an independent miner to pay you in fiat currency, directing your ASIC towards them in the most convenient currency? Bitcoin is not always the most frictionless choice, but even so, it is reasonable to imagine continuing down a path where "hashing" can be performed by many entities of your choice, all done on behalf of a small group of "pools," with the entire network needing their permission to truly record anything on the blockchain.

Who is Actually Hashing?

Let’s look at this issue from a broader context. We have mentioned that some larger participants want to distance themselves from Bitcoin as much as possible, so they are willing to delegate Bitcoin-related activities to their pools as much as possible. These pools are open to regulation, and their massive hash power is very comfortable with this.

This again introduces economic irrationality from the perspective of the network itself, manifested in the behavior of mining blocks that meet certain arbitrary standards. When this has happened in the past, it did not last long due to strong community opposition and the absurdity of trying to actively please a constantly changing regulatory regime in a jurisdiction without being asked to do so. But in fact, this is a choice that exposes the risks of centralized block template building. Would a miner in one jurisdiction attempt to ban or refuse to process transactions from another jurisdiction? Are miners merely extensions of government or influential bad actors? There are concrete examples showing that some pools refuse transaction fees to profit illegally, sometimes just to comply with regulatory pressure. From the network's perspective, this again appears economically irrational.

The most extreme recent example is a transaction fee of 19 BTC paid in a single transaction, ultimately discovered by F2Pool, which was ostensibly an error. As an FPPS pool, they became the custodians of this 19 BTC mining fee and chose to return it to the person who made the mistake. This perfectly illustrates the cost of placing an oversized intermediary between miners and the Bitcoin network. In a PPLNS pool, the likelihood of this happening is lower. Not because PPLNS pools are necessarily trustless or non-custodial, but because it is more difficult for pools to attempt this since they may have already deposited the share of the mining funds earned into the miners' accounts internally, causing greater backlash.

While there is no difference in principle, it is worth comparing what would happen if a pool paid its miners in the coinbase/generated transaction. In that case, the money is already in the miners' hands, and the pool cannot intercept fee income. Therefore, in this example, the pool wants to appear generous or fair, yet it causes its miners to lose $500,000 in fee income, a position it should not have made a decision about.

The Next Issue: 51% Attacks and Other Attacks

This should be easy to explain: everyone currently knows what a 51% attack is. However, far less understood is that (until the network bypasses it) the 51% requirement is what guarantees this attack method can succeed and persist, rather than being merely destructive.

In fact, any entity with over 20% of the network share can trigger problems through various attack methods, some of which are executed in the wild but are rarely discussed, which I will elaborate on later. But before that, we can see that only two entities in the network reliably exceed 51% of total hash power. Worse, one of the largest pools has not carefully concealed the additional 10% of blocks discovered through another large pool, which maintains a strategic partnership with its parent company. The fact that this farce continues is unconvincing.

There are two common responses to this. First, people point out that miners can simply switch pools to vote if they band together for a 51% attack. Second, any pool attempting to do so is crazy, for the simple reason that destroying Bitcoin will lead to a price drop, and anyone invested in the ecosystem does not want that to happen. The second argument ignores human history and further assumes that people will never be forced into destructive behavior, thus causing destruction merely for the sake of it or for other malicious purposes. It also does not consider that the market does not necessarily always reflect a good indicator of problems with Bitcoin, as seen in the hard fork wars of 2017.

However, the first argument is more solid, as it assumes that in the case a pool indeed becomes too large, miners will always switch to other pools. In reality, if a pool attempts to do so, reality will intervene, and we will realize that despite building 99% of block templates, pools are not actually miners. As a famous case study, Ghash.io caused a fatal downward spiral due to exceeding 40% of hash power.

Thus, we have proven that this is not actually a problem; miners can trust that they will switch to other pools. In fact, if large mining operations are bound by red tape, this assumption becomes less reliable, but let’s at least assume we are fairly confident that such an attack is unlikely to occur.

Unfortunately, the realization that any pool's hash power once it exceeds a dreadful threshold will migrate elsewhere leads to self-regulation. But this does not help, as they do not need to actually maintain hash power below the threshold; they only need to create the impression that they do. This is essentially equivalent to accepting all the hash power they can get while forwarding it to other pools to avoid alerting the world to the chaos they could cause.

Thus, our understanding of the network's situation is unclear. 30% of blocks can be openly found by the largest pool and accepted by everyone, while another 10% of total network hash power still points to that pool, just secretly redirected to one or more smaller pools. The miners responsible for that 10% may not realize how it is being used, and it is harder to detect with stratumV2. I will elaborate on this later.

When considering that this redirected hash power can be exploited to harm smaller pools through block mining attacks, this already undesirable situation becomes worse.

Specifically, attackers primarily participate in the mining process as normal users of the victim pool. As a result, they receive a portion of the rewards from any blocks found by the pool as expected. The rewards ultimately flow to the attackers, who can then pay the actual miners without losing any funds. So far, the only damage caused is a false impression of the pool's hash power, making it seem smaller than it actually is, but the smaller pools remain unharmed.

Now, if the attackers decide not to inform the victim pool when they find a block, damage will occur. This will make the victim pool appear unlucky. They seem to find fewer blocks than they should and pay rewards to more participants who are actually mining honestly, which inevitably leads to losses, assuming they do not compensate for these losses in other ways.

If FPPS pools are attacked in this manner, they must burn income, paying miners out of pocket to make up the difference. If it is a PPLNS pool, miners will wonder why they did not receive what they should have. Either way, block mining attacks are anti-competitive and can potentially destroy the victim pool by giving it a bad reputation.

From the perspective of attacking pools, assume they account for 5% of the victim pool's hash power. This means they still receive 95% of the expected income, while the pool appears to be 5% less lucky than expected. This is enough to easily destroy the pool, while in larger pools, the loss of 5% of redirected hash power will be far less significant. If it only represents 1% of the total hash power of a larger pool, then the attacker only loses 5% of the expected reward of 1% - 0.05%. This is a clear advantage for any malicious, comparably sized pool willing to act unethically.

The smaller the pool, the more vulnerable they are to this type of attack. The larger the pool, the more likely they are to block smaller competitors. As large pools approach levels of total hash power that begin to cause community panic, this risk increases, further prompting them to at least hide their hash power in smaller pools, even if they are not actually using it to attack or the frequency of attacks is not high enough for the issue to ultimately be reduced to variance. Since the network provides more consistent payments, larger pools have already enjoyed reduced variability, meaning they can operate within tighter margins, allowing them to charge their miners lower fees. From the perspective of each unattacked miner/pool, this attack means they will enjoy lower difficulty, as the Bitcoin network will adjust to accommodate fewer total blocks.

Is block withholding merely theoretical? Absolutely not. Even in early 2015, several pools suffered attacks in this manner. Preventing such attacks is very difficult because pools must monitor all miners and thoughtfully decide whether to kick them out of the pool and/or refuse to pay them if their luck reaches statistically impossible levels, which pools can reasonably assume indicates malicious behavior. Such attacks also incentivize pools to "know their miners" and regulate payments, which certainly makes life more difficult for those who wish to mine without permission.

In any case, the overall effect of all this is that people are more willing to mine with larger pools for another reason.

We have heard large miners publicly state that they are abandoning smaller pools because the payments they receive do not meet expectations.

This is highly undesirable because larger pools and the larger miners using them are more likely to be burdened by regulatory scrutiny, making them more likely to engage in actions that harm Bitcoin, even beyond the centralization of block templates and the temporary custody of all block rewards.

Pools effectively become agents, imposing bureaucratic nonsense "on behalf of" their miners. Currently, the two largest pools require users to jump through a bunch of cumbersome steps, including those that expose their identities, which should not and must not be something people mining Bitcoin outside of independent mining have to go through.

On the final point of block withholding, aside from threatening to make life more difficult for smaller pools and those wishing to mine with them, I say to those who may still try to view this as purely theoretical: even if it has clearly happened in the past, do we think pools can naturally maintain a consistent and obviously tolerable scale? This implies that new hash power coming online is always somewhat evenly distributed. We must believe that a pool can suddenly appear, grow rapidly, and then stop near the threshold required before people feel uneasy. Do we see pools requesting people to stop mining with them, or simply limiting account creation and kicking out miners who exceed the allowed hash power within existing accounts? Of course not.

The two more likely scenarios are that miners are collectively self-regulating. But this is less likely because it is well known that mining with smaller pools means miners earn less Bitcoin, even if the reasons I have presented in this article do not fully explain why. Not to mention that every time there is a mass migration of quality from one pool, it is very noticeable, or that pools simply mislead people about the hash power they point to.

In addition, smaller pools have another problem: they may go days without finding a block, while larger pools will not exceed a few hours. This is a resolution issue; the higher your hash power, the closer you get to expectations in the short term. Unfortunately, this leads to a lower limit, below which pools cannot expect to make up for losses during bad luck, at which point it becomes impossible to compete.

The two-week period between difficulty epochs means that enough blocks must be found during this two-week period for any bad luck to have a chance to be balanced out by subsequent good luck. If not, for example, if a pool's expected block rate is one block every 13 days and they do not find a block before the upward difficulty adjustment causes them to drop to a projection of one block every 15 days, then the previous time window will forever close. If it is a PPLNS pool, miners' income will be less than they might have otherwise received. If it is an FPPS pool, the pool will burn a lot of cash and/or go bankrupt.

This means only so many pools can exist, at least according to how pools operate today. Simply put, it is impossible to have hundreds of them, as many of them may collapse during adverse times, and due to having less than 1% of the network hash power, they may not even reliably find a block every day, facing periods that could last weeks without a block. This is a limitation imposed on us by Bitcoin itself.

How Do Miners and Pools Communicate?

The communication protocol between miners and pools is Stratum (which is gradually being replaced by StratumV2). StratumV1 is both old and deeply flawed. First, all communication is transmitted in plaintext. This means that Internet Service Providers not only know you are mining but also understand the scale of your mining. Anyone who can eavesdrop on your network traffic can perform man-in-the-middle attacks, leading to your computer and hash power being used for someone else's mining. This has previously been abused by unknown attackers to steal hash power away from the intended pools.

Aside from some inefficiencies, StratumV1 also fails to provide miners with a practical way to build their own block templates while still mining in pools. All these issues are addressed in the highly attractive StratumV2 (originally called "GBT," then "Better Hash"), which we will return to later.

Hardware/Firmware

Before discussing the dynamics between pools and miners, we will deviate a bit because this article would be incomplete if we did not mention that only two companies manufacture ASICs at any meaningful scale: Bitmain and MicroBT. There are other companies, but in reality, almost all hashing occurs on machines made by these two companies.

This is clearly not good, as the root cause is that chip manufacturing is very difficult, leading to excessive centralization.

The scope of this article does not include discussing solutions here, but there are efforts underway to make home mining more practical (in North America, the main issues are the need for 220-240 volts and dealing with loud noise). Those dedicated to "pleb-mining" projects argue that if enough ordinary Bitcoin users can do this, they can begin to account for a significant proportion of the network's total hash power, which is preferable for most large-scale mining operations, as they are more susceptible to regulatory intervention.

The difficulty of this task lies in the fact that the firmware is closed source. Even custom firmware that can "jailbreak" ASICs is often closed source to ensure that users pay for development costs, meaning the market firmware costs you use represent the mining team's expenses.

The factory firmware on ASICs, especially Bitmain's, clearly demonstrates how confident they have become in their market dominance. Besides being closed source, it is evidently malicious. When starting an Antminer, you are forced to mine for them, although miners can at least prevent this by blocking connections or installing market firmware, then paying development fees, but these fees cannot be avoided; otherwise, miners will refuse to mine. Bitmain has been caught multiple times adding malicious backdoors to its miners' firmware (see Antbleed) and actively working to block market firmware developers.

In fact, the actions of the factory firmware are shocking and clearly highlight the urgent need for competition in the ASIC manufacturing field.

Would anyone feel comfortable if the network rules were enforced by closed-source Bitcoin nodes? Furthermore, imagine if we all knew that these nodes would cause users to lose Bitcoin to the developers of that software; would anyone accept that? In mining, almost no one cares about the sovereignty of the participants. Of course, the importance of node software and ASIC firmware is not equal, and we certainly care more about the former, as we should, but the latter is not unimportant and is clearly being overlooked.

Having said all this, let’s continue to discuss some solutions, particularly focusing on expanding the possibilities for miners and improving existing models.

P2POOL

On this point, there is not much to say other than that it essentially decentralizes every aspect of pooled mining. While it does many ideal things on a small scale, it requires each user to download, verify, and track every other user's shares and prove to each other that they are correctly calculating everything in the template. Achieving this in any adversarial environment is essentially an impossible task. Due to the fundamental nature of pooled mining, the resources required are far greater than those needed to run a full Bitcoin node, not to mention making the miners' work more complicated.

For these reasons, most people overlook it, and only more technical users or idealists tend to use it, understandably, as they cannot bring themselves to use other alternative methods for mining.

STRATUMV2

This is undoubtedly the easiest problem to solve, as it provides practical solutions to many of the issues mentioned in this article.

First, by allowing encrypted communication between pools and miners, ISPs and any other entities that can access your network traffic will no longer easily realize that you are mining or the extent of your mining. Therefore, "MITMing" you to hash on behalf of an attacker also becomes impossible, or at least much less easy.

Second, and perhaps most importantly, it also allows miners to build their own block templates, so while pools will still be trusted coordinators for reward distribution and may still be custodians of block rewards, this will represent a shift of power from pools to miners, which is undoubtedly a good thing.

In a world standardized by StratumV2, miners would be eager to build their own templates, ideally with pools incentivizing miners who build their own templates, which would lead to a stronger Bitcoin.

The community is largely united in its commitment to upgrading the mining ecosystem to StratumV2, but historically, due to the extra effort (though trivial compared to p2pool) and lack of incentives, miners have generally avoided using these solutions.

Regardless of whether StratumV2 exists, there is significant room for improvement. What is needed is a mining pool that provides miners with the ability to directly custody their coins while mining. This requires a pool (or its miners) to build a block template in which miners' rewards are directly paid in the coinbase/generation transaction included in each block. Under the FPPS system, this is impractical in practice, meaning any pool implementing this method would face some miners' reluctance, but those transitioning miners would enjoy greater transparency, as Bitcoin itself would be paid directly to them above a certain threshold, with easily verifiable subsidy and fee income distribution. This could be combined with pools, allowing miners to at least understand the block templates built for them before solving blocks, and after StratumV2, simply verifying that all miners built templates that accurately reflect reward distribution without requiring all miners to constantly perform this operation.

Pools could also address the issue of miners' reluctance to build their own templates by incentivizing them, such as by charging them lower fees. It seems that if miners are still unwilling to bear this burden once it becomes practically feasible again, such additional incentives may be necessary.

The suggestions above would greatly improve the situation.

Many new plans and announcements regarding ASIC manufacturing and mining pool infrastructure are emerging, which should be welcomed developments for anyone hoping to ensure that mining trends towards greater decentralization.