[Signature pagefollows] 2

EXHIBIT 10.6

August 11, 2010

Anchorage Advisors, L.L.C.

610 Broadway, 6th Floor

New York, N.Y. 10012

Anchorage Capital Master Offshore, Ltd.

610 Broadway, 6th Floor

New York, N.Y. 10012

Ladies and Gentlemen:

Reference is made to the Second Amended and Restated Investment Agreement dated as of August 11, 2010 (as amended, supplemented or otherwise modified from time to time, the “Investment Agreement”) by and among Carlyle Global Financial Services Partners, L.P., ACMO-HR, L.L.C. (the “Investor”) and Hampton Roads Bankshares, Inc. (the “Company”). Capitalized terms used herein and not otherwise defined shall have the meanings given to them in the Investment Agreement. The Company and Anchorage Capital Master Offshore, Ltd. (the “Fund” and, together with the Investor, “Anchorage”) hereby agree as follows:

On the First Closing Date, the Company will issue to the Anchorage Anchor Investor (i) a warrant to purchase a number of shares equal to 2% of the sum of (A) the outstanding Common Stock of the Company as of the First Closing Date immediately after giving effect to the Closing, (B) the number of shares of Common Stock to be issued by the Company pursuant to the Rights Offering and pursuant to any sale to the CapGen Investor of any unsubscribed shares in the Rights Offering, and (C) the number of shares of Common Stock to be issued at or simultaneously with the Second Closing pursuant to the Transaction Documents, subject to adjustment as set forth therein, which warrant shall be exercisable immediately upon issuance thereof (the “Anchorage Closing Warrant”); and (ii) a warrant to purchase a number of shares equal to 1% of the sum of (A) the outstanding Common Stock of the Company as of the First Closing Date immediately after giving effect to the Closing, (B) the number of shares of Common Stock to be issued by the Company pursuant to the Rights Offering and pursuant to any sale to the CapGen Investor of any unsubscribed shares in the Rights Offering, and (C) the number of shares of Common Stock to be issued at or simultaneously with the Second Closing pursuant to the Transaction Documents, subject to adjustment as set forth therein, which warrant shall be exercisable upon the earlier of (a) the written stay, modification, termination, or suspension by the Federal Reserve Bank of Richmond and the Virginia Bureau of Financial Institutions of the Written Agreement and (b) the occurrence of a Sale Event (as defined in such warrant) (the “Anchorage Contingent Warrant” and together with the Anchorage Closing Warrants, the “Anchorage Warrants”). Each such Anchorage Warrant shall be substantially in the form attached hereto as Exhibit A-1 and as Exhibit A-2, respectively. The Anchorage Warrants will be issued as an inducement for Anchorage’s participation in the transactions contemplated by the Transaction Documents.

For the avoidance of doubt, (i) the issuance of the Anchorage Warrants shall not have any effect on the amount of the expenses of the Anchorage Anchor Investor and its Affiliates to be payable by the Company pursuant to the Investment Agreement, and (ii) the parties hereto agree that the letters dated as of May 23, 2010 and June 30, 2010, between the Company and Carlyle, regarding the same subject matter, shall be superseded by this letter agreement and shall have no further effect after this letter agreement has been executed and delivered by the parties hereto.

The Company hereby agrees and acknowledges that the registration rights set forth in the Investment Agreement are applicable to the Anchorage Warrants and the Common Shares issuable upon exercise of the Anchorage Warrants, each of which shall be deemed to be “Registrable Securities” thereunder, and the Investor shall be deemed to be a “Holder” for purposes of such registration rights.

The Company and Anchorage hereby agree to make, no later than the Effectiveness Deadline, any revisions to the Anchorage Warrants acceptable to Anchorage and necessary in order to permit a public distribution thereof, including entering into a customary warrant agreement, appointing a warrant agent and taking such other steps as will be needed to facilitate such a public distribution.

For the convenience of the parties hereto, this letter agreement may be executed in any number of separate counterparts, each such counterpart being deemed to be an original instrument, and all such counterparts will together constitute the same agreement. Executed signature pages to this letter agreement may be delivered by facsimile and such facsimiles will be deemed as sufficient as if actual signature pages had been delivered.

This letter agreement will be governed by and construed in accordance with the Laws of the State of New York applicable to contracts made and to be performed entirely within such State.

[Signature page follows]

2

If you agree with the foregoing, please sign and return a copy of this letter, which will constitute our agreement with respect to the subject matter of this letter.

3