MIDWESTONE FINANCIAL GROUP, INC. FIFTH AMENDMENT TO SECOND AMENDED ANDRESTATED CREDIT AGREEMENT AND FIRST AMENDMENT TO TERM NOTE

Exhibit 10.5.1

MIDWESTONE FINANCIAL GROUP, INC.

FIFTH AMENDMENT TO SECOND AMENDED AND RESTATED

CREDIT AGREEMENT AND FIRST AMENDMENT TO TERM NOTE

This Fifth Amendment to Second Amended and Restated Credit Agreement and First Amendment to Term Note (herein, the “Amendment”) is entered into as of November 27, 2006, between MidWestOne Financial Group, Inc., an Iowa corporation (the “Borrower”), and Harris N.A. (the “Bank”).

PRELIMINARY STATEMENTS

A. The Borrower and the Bank are parties to that certain Second Amended and Restated Credit Agreement, dated as of November 30, 2003, as amended (the “Credit Agreement”). All capitalized terms used herein without definition shall have the same meanings herein as such terms have in the Credit Agreement.

B. The Borrower has requested that the Bank extend the amortization and final maturity of the Term Loan, and the Bank is willing to do so under the terms and conditions set forth in this Amendment.

NOW, THEREFORE, for good and valuable consideration, the receipt and sufficiency of which is hereby acknowledged, the parties hereto agree as follows:

SECTION 1.    AMENDMENTS.

Subject to the satisfaction of the conditions precedent set forth in Section 2 below, the Credit Agreement shall be and hereby is amended as follows:

1.1. Section 1.2 of the Credit Agreement (Term Loan) shall be and hereby is amended and restated in its entirety to read as follows:

Section 1.2. Term Loan. Subject to the terms and conditions hereof, the Bank agrees to make a term loan (the “Term Loan”) to the Borrower concurrently herewith in the principal amount of $6,000,000. The Term Loan shall be made against and evidenced by a promissory note of the Borrower in the form (with appropriate insertions) attached hereto as Exhibit B (the “Term Note”). The Borrower shall make semi-annual principal installment payments on the Term Loan on the last day of May and November of each year, commencing May 31, 2004, with the amount of each such principal installment to equal the amount set forth in Column B below shown opposite of the relevant due date as set forth in Column A below:

-1-

, it being agreed that the final payment of both principal and interest not sooner paid on the Term Loan shall be due and payable on November 30, 2009, the final maturity thereof.

1.2. The first paragraph of Exhibit B to the Credit Agreement and of the Term Note shall each be amended and restated to read as set forth below:

FOR VALUE RECEIVED, the undersigned, M1DWESTONE FINANCIAL GROUP, INC., an Iowa corporation (the “Borrower”), promises to pay to the order of HARRIS N.A. (the “Bank”) at its office at 111 West Monroe Street, Chicago, Illinois, the principal sum of Six Million and no/100 Dollars ($6,000,000.00) in consecutive semi-annual principal installments in amounts and on dates set forth in Section 1.2 of the Credit Agreement hereinafter referred to, with a final installment in the amount of all principal not sooner paid due on November 30, 2009, the final maturity hereof.

1.3. In order to reflect the amendment to the Term Note made pursuant to subsection 1.2 above, the Bank shall type on the Term Note the following legend:

This Note has been amended as provided for in that certain Fifth Amendment to Second Amended and Restated Credit Agreement and First Amendment to Term Note dated as of November 27, 2006.

-2-

; provided, however, that the failure of the Bank to type such a legend shall not affect or impair the effectiveness of the amendment to the Term Note made pursuant to subsection 1.2 above. By signing in the space provided for that purpose below, the Borrower hereby confirms its promise to pay the principal of and interest on the Term Note as amended by this Amendment.

SECTION 2.    CONDITIONS PRECEDENT.

The effectiveness of this Amendment is subject to the satisfaction of all of the following conditions precedent:

2.1. The Borrower and the Bank shall have executed and delivered this Amendment.

2.2. Legal matters incident to the execution and delivery of this Amendment shall be satisfactory to the Bank and its counsel.

SECTION 3.    REPRESENTATIONS.

In order to induce the Bank to execute and deliver this Amendment, the Borrower hereby represents to the Bank that as of the date hereof, after giving effect to amendments set forth above, the representations and warranties set forth in Section 5 of the Credit Agreement are and shall be and remain true and correct (except that the representations contained in Section 5.5 shall be deemed to refer to the most recent financial statements of the Borrower delivered to the Bank) and the Borrower is in compliance with the terms and conditions of the Credit Agreement and no Default or Event of Default exists under the Credit Agreement or shall result after giving effect to this Amendment.

SECTION 4.    MISCELLANEOUS.

4.1. The Borrower heretofore executed and delivered to the Bank various Collateral Documents. The Borrower hereby acknowledges and agrees that the Liens created and provided for by the Collateral Documents continue to secure, among other things, the Obligations arising under the Credit Agreement as amended hereby; and the Collateral Documents and the rights and remedies of the Bank thereunder, the obligations of the Borrower thereunder, and the Liens created and provided for thereunder remain in full force and effect and shall not be affected, impaired or discharged hereby. Nothing herein contained shall in any manner affect or impair the priority of the liens and security interests created and provided for by the Collateral Documents as to the indebtedness which would be secured thereby prior to giving effect to this Amendment.

4.2. Except as specifically amended herein, the Credit Agreement shall continue in full force and effect in accordance with its original terms. Reference to this specific Amendment need not be made in the Credit Agreement, the Notes, or any other instrument or document executed in connection therewith, or in any certificate, letter or communication issued or made

-3-

pursuant to or with respect to the Credit Agreement, any reference in any of such items to the Credit Agreement being sufficient to refer to the Credit Agreement as amended hereby.

4.3. The Borrower agrees to pay on demand all costs and expenses of or incurred by the Bank in connection with the negotiation, preparation, execution and delivery of this Amendment, including the fees and expenses of counsel for the Bank.

4.4. This Amendment may be executed in any number of counterparts, and by the different parties on different counterpart signature pages, all of which taken together shall constitute one and the same agreement. Any of the parties hereto may execute this Amendment by signing any such counterpart and each of such counterparts shall for all purposes be deemed to be an original. Delivery of executed counterparts of this Amendment by telecopy shall be effective as an original. This Amendment shall be governed by the internal laws of the State of Illinois.

[SIGNATURE PAGE TO FOLLOW]

-4-

This Fifth Amendment to Second Amended and Restated Credit Agreement is entered Into as of the date and year first above written.

Accepted and agreed to:

-5-