: How to Program AWS Spot Instances to Strategically Bid So the Auction is Never Lost Until a Competitor Beats the Maximum I'm Willing to Pay? I believe I'm in the right section of stack exchange

@Odierno851

There is no secret methodology to be determined here, and no need to do what you are trying to do.

You should bid what the machines are worth to you, because you will not pay any more than the market demands.

Now, I previously believed Amazon would just charge me for the highest price right above the next lowest competitor automatically ...

I'm not sure where you got that idea.

There is a fixed spot price at any moment in time for all instances with all four of the following identical configuration elements:


same availability zone
same instance class
same platform (Linux/Unix, Windows, or SUSE)
same product class (VPC or EC2 Classic)



“Everyone pays that same Spot Price for that period regardless of whether their maximum bid price was higher”

— docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html

In light of that, you are either mistaken with the following assertion, or you need to contact AWS support, because this is incorrect:

They charge me for whatever I bid even when I know there is no one else around to counter bid me.

That's not correct, on two counts.


“You will often pay less per hour than your maximum bid price. The Spot Price is adjusted periodically as requests come in and the available supply of instances changes. ”

— docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html

My bills are consistent with the documentation in this regard. I am consistently being billed the spot market price as shown in the console, which is most definitely not my max bid.

If you aren't seeing this, then I suspect you are not interpreting market signals correctly.

Importantly, you don't have any way of knowing who else or whether someone else is around to counter bid you, since the current market price does not reflect the current high bid, and in cases where supply exceeds demand, I have found that the presence or absence of my instances does not affect the market price.

The only conclusion I can draw from my experience analyzing pricing history and trends is that EC2 has a minimum price for each combination of the four factors listed above; when more of that particular capacity exists in a particular place than there are total current spot requests, then the spot market price remains at the minimum price for that instance class + availability zone + platform + product. Otherwise, it rises.

Your spot requests are probably shifting the market by their presence. If the market is not at its minimum price, then you can be almost certain that this is what you're seeing -- in which case, it would look like they're charging you what you bid, but that's because your requests are causing the minimum price to rise.

Let's assume -- with numbers purely for the sake of illustration -- the following:

You and I are both bidding for c3.large instances, running Linux/Unix, in Amazon VPC, in the "us-east-1a" availability zone. (Let's further assume, for the sake of discussion, that "us-east-1a" refers to the same availability zone in my AWS account and your AWS account, which is by no means necessarily true -- availability zone to account mappings are pseudo-random among different AWS customers.)

We'll speculate that EC2 has only 10 such instances available for spot allocations, and the minimum market price is [CO].032/hour. My max bid is [CO].096/hour.

I have 8 of these running, and another customer has 1 instance running, and the other customer has a max bid of [CO].035/hour.

The other customer and I are both paying [CO].032/hour, since there are still idle machines allocated for servicing the spot market, so our instances have not altered the pricing.

But if you are watching the price, you would have no sense of whether our machines are running or not, or whether we have bid against you, since you don't see the price changing.

Now, you come along, see the price at [CO].032/hour, and you bid [CO].04 for 2 instances.

There's not enough capacity to serve 8 + 1 + 2 = 11 because only 10 machines are available, so the system decides who gets to run and who has to stop, and the market shifts.

My max bid of [CO].096 is the highest, so my 8 machines get to keep running. Your max bid is next in line at [CO].04, so your 2 instances gets started, and the other customer, who has a max bid lower than both of us, gets terminated.

And what happens to the market price?


“the last fulfilled bid, sets the Spot Price”

— docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-pricing-interruptions.html

The spot price is now, according to the documentation, [CO].04.

Later, I scale back and don't need all of my instances, so I terminate 5 of them. Now there is excess capacity, so the other user (mentioned before, max bid [CO].035) will fire back up, your instances will keep running, and the market reverts to the minimum price of [CO].032 -- which is what we all pay until the market shifts again.

Any time the prices are not at the baseline, the demand has exceeded the supply, and any successful bid is likely to raise the rate for everyone.

The spot market is a fundamentally fair marketplace. Everyone pays the price bid by the lowest bidder whose request can be fulfilled, except, apparently, in cases where the allocated capacity isn't being fully utilized, in which case, a minimum price established for that particular configuration is what appears to be applied.

The only real "secrets" are finding the particular location/configuration combinations that work for you that are also in the least demand, and bidding what those machines are actually worth to you. You won't pay more than that, and sometimes you will pay less.

It would also, apparently, make sense not to put multiple machines on the same spot requests, but to stagger you bid amounts if you need multiple identical machines, but you don't necessarily need all of them running... because if you are right on the edge of the demand curve, it appears that your lowest fulfilled bid could be, in some cases, the bid that sets the market price... so, in the example above, if you had bid [CO].04 and [CO].036, the market would have coalesced at [CO].036 instead of [CO].04, saving us all just a little bit per hour.

Vote Up Vote Down