AMENDMENT NO. 2 TO NOTE AGREEMENT

Exhibit 10.40

AMENDMENT NO. 2

TO NOTE AGREEMENT

THIS AMENDMENT NO. 2 (the “Amendment”) to the Note Agreement dated as of April 18, 2011 (as previously amended by Amendment No. 1 thereto dated December 21, 2011, the “Agreement”), by and among MHI Hospitality Corporation, a Maryland corporation (the “Borrower”), and each lender a party thereto from time to time (together with their successors and assigns, each, a “Lender”, and collectively, the “Lenders”) and Essex Equity High Income Joint Investment Vehicle, LLC, a Delaware limited liability company, as agent for the Lenders (the “Agent”, and collectively with the Borrower and the Lenders, the “Parties”), is made and entered into by and among the Parties as of June 15, 2012 (subject to the conditions precedent to effectiveness set forth in Section 2 hereof, the “Amendment Date”). Capitalized terms used herein and not otherwise defined shall have the meanings set forth in the Agreement.

WHEREAS, the Parties desire, for their mutual convenience and benefit, to amend the Agreement as set forth in this Amendment.

NOW, THEREFORE, in consideration of the mutual covenants and agreements contained herein, and other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the Parties hereby agree as follows:

1. Upon the making of the Prepayment (as defined below), the Parties agree that the aggregate outstanding balance of the Loans (including, without limitation, accrued and unpaid interest) shall be zero and that the amount of undrawn Term Loan Commitments under the Agreement shall be $7,000,000, and references in the Agreement to the maximum aggregate principal amount of all Loans and to the Term Loan Commitments shall be interpreted accordingly. Of such undrawn Term Loan Commitments, $2,000,000 shall be solely reserved for Loans the proceeds of which the Borrower shall use to cause its Subsidiary to repay principal amounts outstanding under the existing mortgage on the Crowne Plaza Jacksonville Riverfront Hotel Property. For the avoidance of doubt, the Borrower’s ability to borrow, and the Lenders’ obligation to make, Loans up to the Term Loan Commitments shall be governed by the terms and conditions of the Agreement as amended hereby.

2. Section 7 of the Agreement is hereby amended by adding the following sentence at the end thereof:

“No part of the proceeds of the Loans borrowed hereunder will be used for the purpose of redeeming shares of Preferred Stock.”

3. The effectiveness of this Amendment is subject to and contingent upon the fulfillment of each and every one of the following conditions:

(a) the Borrower or one of its Subsidiaries shall have closed on a financing transaction with C1 Bank, as lender, in respect of the hotel property known as the Crowne Plaza Tampa Westshore (the “Tampa Financing”);

(b) the Borrower shall have made or caused to be made, from the proceeds of the Tampa Financing, (i) a prepayment of outstanding Loans in an amount of not less than $1,500,000 (the “Prepayment”) and (ii) payment of all accrued and unpaid interest with respect to the Prepayment; and

(c) the redemption of certain issued and outstanding shares of the Preferred Stock held by Essex Illiquid, LLC and Richmond Hill Capital Partners, LP (together the “Preferred Stockholders”) in the manner contemplated in that certain Agreement, Waiver and Consent by Preferred Stockholders dated as of June 15, 2012 shall have occurred.

4. This Amendment shall be deemed to be incorporated into the Agreement and made a part thereof. All references to the Agreement in any other document (including, without limitation, that certain promissory note made and delivered by the Borrower to the Agent dated as of April 18, 2011) shall be deemed to be references to the Agreement as modified by this Amendment. Except as specifically amended or modified herein, the Agreement shall continue in full force and effect and shall be enforceable in accordance with each of the terms thereof. To the extent that the terms of this Amendment conflict with the terms of the Agreement, the terms of this Amendment shall be deemed to govern.

5. This Amendment may be executed simultaneously in one or more counterparts, and by different Parties in separate counterparts, each of which when executed will be deemed an original, but all of which taken together will constitute one and the same instrument.

6. This Amendment will be governed by, and construed in accordance with, the laws of the state of New York applicable to contracts executed in and to be performed entirely within such jurisdiction, without reference to conflicts of laws provisions.

7. This Amendment may not be amended, modified or waived except by an instrument in writing signed on behalf of each of the Parties.

[Signature page follows.]

2

IN WITNESS WHEREOF, the Parties, intending to be legally bound, have executed this Amendment as of the Amendment Date.

[SIGNATURE PAGE TO AMENDMENT NO. 2 TO NOTE AGREEMENT]

[SIGNATURE PAGE TO AMENDMENT NO. 2 TO NOTE AGREEMENT]