COVENANT NOT TO COMPETE (Innkeepers)

Exhibit 10.4

COVENANT NOT TO COMPETE

(Innkeepers)

THIS COVENANT NOT TO COMPETE (this “Agreement”) is entered as of the 4th day of August, 2003 (the “Effective Date”) by and between INNKEEPERS USA LIMITED PARTNERSHIP, a Virginia limited partnership (the “Partnership”), INNKEEPERS USA TRUST, a Maryland real estate investment trust and the general partner of the Partnership (the “REIT”; the REIT and the Partnership are collectively referred to as the “Company”), INNKEEPERS HOSPITALITY, INC., a Virginia corporation f/k/a JF Hotel, Inc., and JEFFREY H. FISHER (“Mr. Fisher”).

THE PARTIES ENTER INTO THIS AGREEMENT on the basis of the following facts, understandings and intentions:

A. WHEREAS, the REIT desires to take advantage of the provisions of the so-called “REIT Modernization Act” (the “Act”) and create so-called “taxable REIT subsidiaries” to lease certain of its hotels to and simultaneously therewith engage Innkeepers Hospitality Management, Inc. (the “IH Manager”), a Fisher Affiliate (as defined herein), to act as an “eligible independent contractor” (as defined in Section 856(d)(9) of the Internal Revenue Code of 1986, as amended) and manage such hotels pursuant to management agreements, all as permitted by the Act;

B. WHEREAS, in connection with the REIT’s desire to take advantage of the provisions of the Act, simultaneously herewith it is entering into that certain Master Lease Assignment Agreement (the “Assignment Agreement”) by and among the REIT, the Partnership, certain Company Affiliates (as defined herein), Innkeepers Hospitality, Inc., certain Fisher Affiliates and the IH Manager;

C. WHEREAS, it is a condition to the consummation of the transaction contemplated by the Assignment Agreement that this Agreement be entered into by the parties hereto; and

D. WHEREAS, the parties hereto so desire to enter into this Agreement, which amends and restates in its entirety that certain Exclusive Hotel Development Agreement and Covenant Not to Compete dated as of September 30, 1994 by and among the parties hereto (the “Hotel Development Agreement”).

NOW THEREFORE, IN CONSIDERATION of the mutual covenants and promises of the parties provided for in this Agreement and for other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the parties hereto agree as follows:

1. Amendment and Restatement. The parties hereto agree that this Agreement shall amend and restate the Hotel Development Agreement in its entirety.

2. Definitions. The following terms as used in this Agreement shall have the following meanings (applicable to both the singular and plural forms of the terms defined):

a. “Acquisition of Hotel Property” means engaging in the activity of soliciting, seeking to acquire (or obtaining an option or first right of refusal to acquire) and/or acquiring, any interest in Hotel Property.

b. “Affiliate” means (i) any other person or entity directly or indirectly controlling, controlled by, or under common control with the person or entity to which such term applies; or (ii) as to any natural person, such person’s spouse, child, grandchild, sibling, parent, aunt, uncle, or cousin, as well as the spouse of any of the foregoing. In addition, as to any corporation or partnership, any person in control of

such partnership as a general partner or otherwise or in control of such corporation shall be deemed to be an Affiliate of such partnership or corporation. For purposes of this Agreement, “control” as applied to any person or entity means the possession, either directly or indirectly, of the power to direct or cause the direction of the management, policies and decision-making of such person or entity, whether through the ownership of voting interests, by contract or otherwise. “Control” shall also include, without limitation, the possession of direct or indirect equity or beneficial interests in at least fifty percent (50%) of the profits or voting control of any entity.

c. “Company Affiliate” means any Affiliate of the Company.

d. “Fisher Affiliates” means any Affiliate of Mr. Fisher or the IH Manager. The Fisher Affiliates existing as of the Effective Date are set forth on Schedule 1 attached hereto.

e. “Fisher Employment Agreement” means the Employment Agreement between the REIT and Mr. Fisher effective as of February 1, 1997 and amended on November 13, 2000.

f. “Hotel Construction” means engaging in the construction, renovation or repair of improvements on Hotel Property.

g. “Hotel Development Activity” means engaging in directly, through an Affiliate, or being employed by any entity undertaking, or otherwise undertaking to do any of the following with respect to Hotel Property: (i) Acquisition of Hotel Property, (ii) Hotel Construction, (iii) Hotel Entitlements, (iv) Speculation, or (v) Hotel Management and Operation.

h. “Hotel Entitlements” means engaging in the process by which a person with an interest in Hotel Property obtains necessary or desirable governmental approvals, licenses, permits, entitlement or agreements for the commencement of Hotel Construction.

i. “Hotel Management and Operation” means engaging in directly or through an Affiliate, or being employed by any entity undertaking, or otherwise undertaking the day-to-day management and operation of a hotel, whether pursuant to a master lease, management agreement or other arrangement.

j. “Hotel Property” means any Property, whether in fee or leasehold, that is zoned for (or may be zoned for), or is suitable for (or may be suitable for), hotel purposes, including, without limitation, motels, motor inns, extended stay hotels and the like, whether in whole or in part.

k. “Independent Trustee” shall mean a member of the Board of Trustees of the REIT who is defined as an “Independent Trustee” in the charter documents of the REIT.

l. “Property” means any real property or any interest therein located in the United States of America.

m. “Speculation” means engaging in the activity of soliciting, seeking to acquire (or obtaining an option or a first right of refusal to acquire) and/or acquiring, any interest in Property with the intention at any time of acquiring (or obtaining an option or a first right of refusal to acquire) or holding Property for subsequent sale or other transfer to any person for purposes of Hotel Development Activity.

3. Term. The rights granted to the Company hereunder, and the restrictions imposed on Mr. Fisher and the Fisher Affiliates, shall commence as of the Effective Date and shall terminate one (1) year after the occurrence of both (i) Mr. Fisher ceasing to be a director, trustee, officer or employee of the

2

Company and (ii) Mr. Fisher no longer owning any material financial interest in any Fisher Affiliate (or any successor entity) undertaking Hotel Management and Operation for any of the hotels leased to or owned by the Company or Company Affiliates; provided if Mr. Fisher’s employment with the Company is terminated without cause by the Company as provided in Section 10(b) of the Fisher Employment Agreement (other than a Voluntary Termination as defined therein), such one (1) year period shall be reduced to six (6) months.

4. Negative Covenants. During the term of this Agreement:

a. Restrictions. Mr. Fisher, personally or through any Fisher Affiliate, shall not conduct any Hotel Development Activity without the prior written consent of a majority of the Independent Trustees, which consent may be provided or withheld in their sole discretion.

b. Limited Hotel Management and Operations. Notwithstanding Section 4.a, Mr. Fisher, personally or through a Fisher Affiliate, shall be permitted to engage in Hotel Management and Operation, provided that such Hotel Management and Operation shall not be conducted within five (5) miles of any Hotel Property which the Company either owns or has the written contractual right to acquire, whether in fee, by ground lease or otherwise without the prior written consent of a majority of the Independent Trustees, which consent may be provided or withheld in their sole discretion, provided that in furtherance of, and ancillary to, any permitted Hotel Management and Operation, Mr. Fisher, through a Fisher Affiliate, may repair, renovate or make improvements to a Hotel Property on behalf of the party for which Mr. Fisher or a Fisher Affiliate is engaged in such permitted Hotel Management and Operation and may obtain any necessary licenses, permits or approvals in connection therewith.

c. No Beneficial Ownership. Mr. Fisher shall not hold directly or indirectly any beneficial interest in any entity engaged in any Hotel Development Activity, except for any interest in (i) a company traded on a nationally recognized public securities exchange, provided such interest does not exceed five percent (5%) or (ii) an entity engaged in Hotel Management and Operation.

d. Loans. Mr. Fisher shall not directly or indirectly make any loan to, or hold any note evidencing a loan from, any entity engaged in any Hotel Development Activity.

e. Competitive Entity. Mr. Fisher shall not be a director or trustee, officer, or employee of, or consultant to (whether for compensation or not) any entity engaged in any Hotel Development Activity other than a Fisher Affiliate engaged in any Hotel Development Activity permitted by this Section 4.

5. Notification to Independent Trustees. If Mr. Fisher, personally or through a Fisher Affiliate, desires to engage in any Hotel Development Activity, Mr. Fisher shall be obligated to describe fully the proposed activity in a written notice (the “Disclosure Notice”) to the Company and the Independent Trustees. A Disclosure Notice shall only pertain to a specific proposed project and the referenced proposed project shall be described therein with specificity as to timing, location, scope and the extent of involvement by Mr. Fisher financially and in terms of his time commitment. A Disclosure Notice may not request approval for any conceptual or non-project specific activity or to any activity which is prohibited by this Agreement.

6. Primary Responsibility to REIT. The parties hereto expressly agree that Mr. Fisher shall devote substantially all of his time, attention and effort to the REIT’s affairs; provided that, subject to Section 4 hereof, the REIT acknowledges that Mr. Fisher may, from time to time, hold an executive officer position with Fisher Affiliates engaging in Hotel Management and Operation, and that Mr. Fisher may devote business time to these companies so long as such activities do not interfere with the performance of Mr. Fisher’s duties as Chairman of the Board, President and Chief Executive Officer of

3

the REIT. Mr. Fisher shall have full authority and responsibility, subject to the general direction, approval and control of the REIT’s Board of Trustees, for formulating policies and administering the REIT in all respects.

7. Termination of Right of First Refusal. The parties agree that certain Right of First Refusal and Option to Purchase by and among the parties hereto and dated as of September 30, 1994, and all terms and provisions thereof, are hereby terminated simultaneously with the execution of this Agreement and shall be of no further force or effect.

8. Miscellaneous.

a. Complete Agreement; Construction. This Agreement, and the other agreements and documents referred to herein, shall constitute the entire agreement between the parties with respect to the subject matter thereof and shall supersede all previous negotiations, commitments and writings with respect to such subject matter.

b. Governing Law. This Agreement shall be governed by and construed in accordance with the laws of the jurisdiction of the State of Florida without regard to the principals of conflicts of laws thereof.

c. Notices. All notices and other communications required or permitted hereunder shall be in writing, shall be deemed duly given upon actual receipt, and shall be delivered (i) in person, (ii) by registered or certified mail (air mail if addressed to an address outside of the country in which mailed), postage prepaid, return receipt requested, or (iii) by facsimile or other generally accepted means of electronic transmission (provided that a copy of any notice delivered pursuant to this clause (iii) shall also be sent pursuant to clause (ii), addressed as follows (or to such other addresses as follows (or to such other addresses as may be specified by like notice to the other parties):

d. Amendments. No amendment, modification or supplement to this Agreement shall be binding on any of the parties hereto unless it is in writing and signed by the parties in interest at the time of the modification, and further provided any such modification is approved by a majority of the Independent Trustees.

e. Successors and Assigns. Neither this Agreement nor any rights or obligations hereunder shall be assignable by a party to this Agreement without the prior, express written consent of the other party. This Agreement and all of the provisions hereof shall be binding upon and inure to the benefit of the parties to this Agreement and their respective successors and permitted assigns.

f. No Third-Party Beneficiaries. This Agreement is solely for the benefit of the parties to this Agreement and should not be deemed to confer upon third-parties any remedy, claim, liability,

4

reimbursement, claims or action or other right in excess of those existing without reference to this Agreement.

g. Titles and Headings. Titles and headings to sections in this Agreement are inserted for the convenience of reference only and are not intended to be part of or affect the meaning or interpretation of this Agreement.

h. Maximum Legal Enforceability; Time of Essence. Any provision of this Agreement which is prohibited or unenforceable in any jurisdiction shall, as to such jurisdiction, be ineffective to the extent of such prohibition or unenforceability without invalidating the remaining provisions hereof. Any such prohibition or unenforceable in any jurisdiction shall not invalidate or render unenforceable such provision in any other jurisdiction. Without prejudice to any rights or remedies otherwise available to any party to this Agreement, each party hereto acknowledges that damages would not be an adequate remedy for any breach of the provisions of this Agreement and agrees that the obligations of the parties hereunder shall be specifically enforceable. Time shall be of the essence as to each and every provision of this Agreement.

i. Further Assurance. The parties to this Agreement will execute and deliver or cause the execution and delivery of such further instruments and documents and will take such other actions as any other party to the Agreement may reasonably request in order to effectuate the purpose of this Agreement and to carry out the terms hereof.

[signature page follows]

5

IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be duly executed as of the day and year first written above.

6

SCHEDULE 1

FISHER AFFILIATES

Innkeepers Hospitality, Inc.

Innkeepers Hospitality II, Inc.

Innkeepers Hospitality III, Inc.

Innkeepers Hospitality IV, Inc.

Innkeepers Hospitality V, Inc.

Innkeepers Hospitality VI, Inc.

Innkeepers Hospitality VII, Inc.

Innkeepers Hospitality Management, Inc.

Innkeepers Hospitality Florida, Inc.

Innkeepers Hospitality Florida Management, Inc.

7