SECOND AMENDMENT TO LOAN AGREEMENT

Exhibit 10.36

SECOND AMENDMENT TO LOAN AGREEMENT

THIS SECOND AMENDMENT TO LOAN AGREEMENT (this “Amendment”) is made and entered into as of December 31, 2003 by and among CARROLS CORPORATION, a Delaware corporation (the “Borrower”); each of the Lenders which is or may from time to time become a party to the Loan Agreement (as defined below) (individually, a “Lender” and, collectively, the “Lenders”), BANK OF AMERICA, N.A., as Syndication Agent, SUNTRUST BANK, ATLANTA, as Documentation Agent, MANUFACTURERS AND TRADERS TRUST COMPANY, as Co-Agent, and JPMORGAN CHASE BANK, acting as agent for the Lenders (in such capacity, together with its successors in such capacity, the “Agent”).

RECITALS

A. The Borrower, the Lenders and the Agent executed and delivered that certain Loan Agreement dated as of December 19, 2000, as amended by instrument dated as of March 31, 2003. Said Loan Agreement, as amended, is herein called the “Loan Agreement.” Any capitalized term used in this Amendment and not otherwise defined shall have the meaning ascribed to it in the Loan Agreement.

B. The Borrower, the Lenders and the Agent desire to amend the Loan Agreement in certain respects.

NOW, THEREFORE, in consideration of the premises and the mutual agreements, representations and warranties herein set forth, and further good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the Borrower, the Lenders and the Agent do hereby agree as follows:

SECTION 1. Amendments to Loan Agreement.

(a) The following new definitions are hereby added to Section 1.1 of the Loan Agreement:

“Restatement” means the restatement of the consolidated financial statements of the Borrower for fiscal periods ending prior to December 31, 2003 whereby the Subject Transactions have been recharacterized as financing transactions and the Subject Leases have been recharacterized as financing leases on the consolidated financial statements of the Borrower.

“Subject Leases” means the leases for restaurant Properties identified on Exhibit H attached hereto.

“Subject Transactions” means the sale/leaseback transactions entered into with respect to the Subject Leases prior to December 31, 2003.

(b) Section 1.2 of the Loan Agreement is hereby amended to read in its entirety as follows:

1.2 Miscellaneous. The words “hereof,” “herein,” and “hereunder” and words of similar import when used in this Agreement shall refer to this Agreement as a whole and not any particular provision of this Agreement. The term “annualized” as used herein shall mean the multiplication of the applicable amount for any given period by a fraction, the numerator of which is 365 and the denominator of which is the number of days elapsed in such period. This Agreement and the observance, performance and calculation of all covenants hereunder shall be construed as if the Restatement, the recharacterization of the Subject Transactions as financing transactions and the recharacterization of the Subject Leases as financing leases, had not occurred, notwithstanding the effect of the Restatement, including without limitation, treatment of the Subject Transactions as financing transactions and the recharacterization of the Subject Leases as financing leases, in the consolidated financial statements of the Company for future periods. Borrower hereby agrees that in connection with the delivery of Compliance Certificates hereunder it shall provide to the Agent and Lenders a reconciliation from the consolidated financial statements covered by such Compliance Certificate, showing the covenant calculations as if the Restatement, the recharacterization of the Subject Transactions as financing transactions and the recharacterization of the Subject Leases as financing leases had all not occurred and demonstrating compliance with the covenants under this Agreement.

(c) Section 7.2(d) of the Loan Agreement is hereby amended to read in its entirety as follows:

(d) within 100 days after the beginning of each fiscal year, Borrower’s annual business plan for such fiscal year (including its balance sheet and income and cash flow projections for such fiscal year)

(d) A new Exhibit H is hereby added to the Loan Agreement, such new exhibit to be identical to Exhibit H attached hereto.

SECTION 2. Waiver of Prior Defaults. Lenders hereby waive any Events of Default which may have occurred prior to the date hereof as a result of the Restatement, the recharacterization of the Subject Transactions as financing transactions and the recharacterization of the Subject Leases as financing leases on the consolidated financial statements of the Borrower. Lenders hereby further waive any Events of Default arising by reason of the delivery of the Annual Audited Financial Statements of Borrower and Carrols Holdings for the fiscal year 2003 and related Compliance Certificate and Borrower’s annual business plan for the fiscal year 2004 after the date required for such delivery; provided, however, that the failure to deliver such Annual Audited Financial Statements and related Compliance Certificate and Borrower’s annual business plan for the fiscal year 2004 to the Agent and the Lenders on or before May 6, 2004 shall constitute an Event of Default under the Loan Agreement,

SECTION 3. Ratification. Except as expressly amended by this Amendment, the Loan Agreement and the other Loan Documents shall remain unchanged and in full force and effect. None of the rights, title and interests existing and to exist under the Loan Agreement are hereby

2

released, diminished or impaired, and the Borrower hereby reaffirms all covenants, representations and warranties in the Loan Agreement.

SECTION 4. Expenses. The Borrower shall pay to the Agent all reasonable fees and expenses of its respective legal counsel (pursuant to Section 11.3 of the Loan Agreement) incurred in connection with the execution of this Amendment.

SECTION 5. Certifications. The Borrower hereby certifies that except for the Restatement and its effect on the Borrower’s financial statements (a) no material adverse change in the assets, liabilities, financial condition, business or affairs of the Borrower has occurred and (b) after giving effect to this Amendment, no Default or Event of Default has occurred and is continuing or will occur as a result of this Amendment.

SECTION 6. Miscellaneous. This Amendment (a) shall be binding upon and inure to the benefit of the Borrower, the Lenders and the Agent and their respective successors, assigns, receivers and trustees; (b) may be modified or amended only by a writing signed by the required parties; (c) shall be governed by and construed in accordance with the laws of the State of New York and the United States of America; (d) may be executed in several counterparts by the parties hereto on separate counterparts, and each counterpart, when so executed and delivered, shall constitute an original agreement, and all such separate counterparts shall constitute but one and the same agreement and (e) together with the other Loan Documents, embodies the entire agreement and understanding between the parties with respect to the subject matter hereof and supersedes all prior agreements, consents and understandings relating to such subject matter. The headings herein shall be accorded no significance in interpreting this Amendment. Upon the execution of this Amendment by the Borrower and the Majority Lenders, the Loan Agreement shall be deemed to be amended and all references to the Loan Agreement shall mean the Loan Agreement, as amended hereby.

NOTICE PURSUANT TO TEX. BUS. & COMM. CODE §26.02

THE LOAN AGREEMENT, AS AMENDED BY THIS AMENDMENT, AND ALL OTHER LOAN DOCUMENTS EXECUTED BY ANY OF THE PARTIES PRIOR HERETO OR SUBSTANTIALLY CONCURRENTLY HEREWITH CONSTITUTE A WRITTEN LOAN AGREEMENT WHICH REPRESENTS THE FINAL AGREEMENT BETWEEN THE PARTIES AND MAY NOT BE CONTRADICTED BY EVIDENCE OF PRIOR, CONTEMPORANEOUS OR SUBSEQUENT ORAL AGREEMENTS OF THE PARTIES. THERE ARE NO UNWRITTEN ORAL AGREEMENTS BETWEEN THE PARTIES.

3

IN WITNESS WHEREOF, the Borrower, the Lenders and the Agent have caused this Amendment to be signed by their respective duly authorized officers, effective as of the date first above written.

Exhibit H – Subject Leases

The undersigned hereby join in this Amendment to evidence their consent to execution by Borrower of this Amendment, to confirm that each Loan Document now or previously executed by the undersigned applies and shall continue to apply to the Loan Agreement, as amended hereby, to acknowledge that without such consent and confirmation, Lender would not execute this Amendment and to join in the notice pursuant to Tex. Bus. & Comm. Code §26.02 set forth above.

EXHIBIT H

SALE/LEASEBACK

TRANSACTIONS

EXHIBIT H

SALE/LEASEBACK

TRANSACTIONS

EXHIBIT H

SALE/LEASEBACK

TRANSACTIONS