SECOND AMENDMENT TO THE AMENDED AND RESTATED CREDIT AND GUARANTY AGREEMENT

Exhibit 10.3

Execution Copy

SECOND AMENDMENT

TO THE AMENDED AND RESTATED CREDIT AND GUARANTY AGREEMENT

SECOND AMENDMENT, dated as of November 22, 2005 (this “Amendment”), to the Amended and Restated Credit and Guaranty Agreement, dated as of March 5, 2003 (as amended by the First Amendment, dated October 3, 2003, and as further amended, supplemented or otherwise modified from time to time, the “Credit Agreement”), among Houghton Mifflin Company (“HM”), Houghton Mifflin Holdings, Inc. (“Holding”), each lender from time to time party to the Credit Agreement (the “Lenders”), CIBC World Markets Corp. and Goldman Sachs Credit Partners L.P. (“GSCP”), as Joint Lead Arrangers and Joint Bookrunners, GSCP and Deutsche Bank Securities Inc., as Co-Syndication Agents, Canadian Imperial Bank of Commerce, as Administrative Agent and Collateral Trustee, Fleet Securities Inc. and Bank One, N.A., as Co-Documentation Agents, and General Electric Capital Corporation, as Senior Managing Agent.

W I T N E S S E T H:

WHEREAS, HM and Holding have requested certain amendments to the Credit Agreement;

WHEREAS, HM has delivered notice to the Administrative Agent in accordance with Section 2.13 of the Credit Agreement terminating $75,000,000 of the Revolving Commitments, so that the aggregate amount of the Revolving Commitments at the Second Amendment Effective Date is $250,000,000.

NOW THEREFORE, in consideration of the premises herein contained and for other good and valuable consideration, the receipt of which is hereby acknowledged, the parties hereto hereby agree as follows:

1. Defined Terms. Unless otherwise defined herein, terms defined in the Credit Agreement and used herein shall have the meanings given to them in the Credit Agreement.

2. Amendment to Section 1.1.

(a) Section 1.1 of the Credit Agreement is hereby amended by inserting in such Section the following definitions in their appropriate alphabetical order:

“New Term Loan” as defined in Section 6.1(y).

“New Term Loan Commitments” as defined in Section 6.1(y).

“New Term Loan Documentation” as defined in Section 6.1(y).

“New Term Loan Intercreditor Agreement” means the intercreditor agreement executed by the Administrative Agent and the administrative agent for the New Term Loans in order to effectuate the intention of the parties thereto that any New Term Loans be secured on an equal and ratable basis with the Indebtedness incurred under this Credit Agreement.

1

“Second Amendment” means the amendment to this Agreement dated as of November 22, 2005.

“Second Amendment Effective Date” means the date on which the conditions set out in Section 11 of the Second Amendment are waived or satisfied.

“Series” as defined in Section 6.1(y).

(b) The first sentence of the definition of “Permitted Business Acquisition Amount” in Section 1.1 of the Credit Agreement is hereby amended by deleting the text up to and including the word “further” in the first six lines of such definition and inserting in lieu thereof the following:

“Permitted Business Acquisition Amount” shall mean each of $150,000,000 for any individual acquisition and $300,000,000 in the aggregate from the Second Amendment Effective Date until the date of determination;

(c) The definition of “Related Agreement” in Section 1.1 of the Credit Agreement is hereby amended by deleting the period at the end of such definition and inserting in lieu thereof the following:

and the New Term Loan Documentation.

(d) Provision (b) of the definition of “Excess Cash Flow” in Section 1.1 of the Credit Agreement is hereby deleted and replaced with the following:

(b) any permanent voluntary reductions to the Revolving Commitments to the extent that an equal amount of the Revolving Loans simultaneously is repaid, so long as such amounts are not already reflected in Debt Service and any permanent voluntary prepayments of the New Term Loans in accordance with the New Term Loan Intercreditor Agreement,

3. Amendment to Section 2.14. Section 2.14(a) of the Credit Agreement is hereby deleted.

4. Amendment to Section 2.15. Section 2.15(b) of the Credit Agreement is hereby deleted and replaced with the following:

(b) Application of Mandatory Prepayments by Type of Loans. Any amount required to be paid pursuant to Section 2.14(b) shall be applied as follows:

first, to repay any outstanding New Term Loans, if any, to the full extent thereof;

2

second, to repay outstanding Swing Line Loans to the full extent thereof;

third, to prepay the Revolving Loans to the full extent thereof and to further permanently reduce the Revolving Commitments by the amount of such prepayment; and

fourth, to cash collateralize Letters of Credit and to further permanently reduce the Revolving Commitments by the amount of such cash collateralization.

5. Amendments to Section 5.1. Sections 5.1(b), (c), (d) and (e) of the Credit Agreement are hereby amended by deeming all references therein to “Holding and its Subsidiaries” as referring instead to “HM Publishing Corp. and its Subsidiaries”.

6. Amendment to Section 6.1. Section 6.1 of the Credit Agreement is hereby amended by inserting after Section 6.1(x) the following new Section 6.1(y):

; and (y) Indebtedness evidencing the establishment of one or more new term loan commitments (the “New Term Loan Commitments”), by an amount not in excess of $250,000,000 in the aggregate and not less than $75,000,000 initially and thereafter in increments of not less than $20,000,000 (any such loan, a “New Term Loan”), the proceeds of which shall be used for acquisitions; provided that (1) the documentation with respect to such New Term Loan Commitments and New Term Loans (the “New Term Loan Documentation”) shall, except to the extent expressly set forth on Schedule A to the Second Amendment, be substantially identical to the terms of the Credit Agreement and, without limiting the generality of the foregoing, shall not have any representations, warranties, covenants, events of default or any mandatory prepayment provisions which are more onerous to any Credit Party than those contained in the Credit Agreement (unless in each case the corresponding provisions in the Credit Agreement are revised by the Administrative Agent pursuant to its authority conveyed thereunder by Section 17 of the Second Amendment to conform to those relevant provisions in the New Term Loan Documentation) and shall not confer on any holder of the New Term Loan Commitments or New Term Loans more rights than those conferred on the holders of the Indebtedness evidenced by the Credit Agreement and shall otherwise be reasonably acceptable to the Administrative Agent; (2) there shall be no obligor in respect of the New Term Loan Commitments or the New Term Loans which is not an obligor under the Credit Agreement; (3) the Administrative Agent shall have the option in its sole discretion to act as administrative agent with respect to each Series of New Term Loans; (4) no Default or Event of Default under any of the Credit Documents shall exist before or

3

after giving effect to any such Series of New Term Loan Commitments; (5) both before and after giving effect to the making of any Series of New Term Loans, the conditions set forth in Section 3.2(c) and (e) of the Credit Agreement shall be satisfied; (6) Company and its Subsidiaries shall be in pro forma compliance with each of the covenants set forth in Section 6.7 of the Credit Agreement as of the last day of the most recently ended Fiscal Quarter after giving effect to any such Series of New Term Loan Commitments; and (7) Company shall deliver or cause to be delivered any legal opinions or other documents, in each case as reasonably requested by Administrative Agent, in connection with any such transaction; and provided further that any such Indebtedness incurred pursuant to this Section 6.1(y) shall not be otherwise prohibited by the terms of any Contractual Obligation of Holdings, HM or any of its Subsidiaries. Any New Term Loans made pursuant to this Section 6.1(y) shall be designated a separate series (a “Series”) of New Term Loans for all purposes of this Agreement.

7. Amendment to Section 6.2. Section 6.2 of the Credit Agreement is hereby amended by inserting after Section 6.2(v) the following new Section 6.2(w):

; and (w) Liens securing Indebtedness permitted to be incurred pursuant to Section 6.1(y) and subject to the New Term Loan Intercreditor Agreement.

8. Amendment to Section 6.4. Section 6.4 of the Credit Agreement is hereby amended by deleting the period at the end of such Section and inserting in lieu thereof the following:

; and (m) Company or Holdings may redeem, repurchase, retire or defease (including in-substance or legal defeasance) the Senior Notes, the Intermediate Holding Notes, any Permitted Refinancing Indebtedness with respect thereto, or Subordinated Indebtedness; provided that (i) the amount of such redemption, repurchase, retirement or defeasance shall not exceed $100,000,000 in the aggregate and (ii) the Total Leverage Ratio as of the date of determination shall be 0.25 basis points less than the correlative ratio indicated for the relevant period in Section 6.7(b) of the Credit Agreement.

9. Amendments to Section 6.7. Each of Sections 6.7(b), (c) and (d) of the Credit Agreement is hereby amended in its entirety by substituting the following in its place:

(b) Total Leverage Ratio. Holding and Company shall not permit the Total Leverage Ratio as of the last day of any Fiscal Quarter specified below to exceed the correlative ratio indicated:

4

(c) Senior Leverage Ratio. Holding and Company shall not permit the Senior Leverage Ratio as of the last day of any Fiscal Quarter specified below to exceed the correlative ratio indicated:

(d) Maximum Consolidated Capital Expenditures. (i) Company shall not, and shall not permit its Subsidiaries to, make or incur Consolidated Capital Expenditures, in any Fiscal Year specified

5

below, in an aggregate amount in excess of the corresponding amount set forth below opposite such Fiscal Year; provided, such amount for any Fiscal Year shall be increased by an amount equal to 50% of the excess, if any (without giving effect to any adjustment in accordance with this proviso) over the actual amount of Consolidated Capital Expenditures for the previous Fiscal Year:

10. Amendment to Schedule 1.1(a). Footnote (e) to Schedule 1.1(a) of the Credit Agreement is hereby deleted and replaced with the following:

(e) Not to exceed $35.0 million, of which $25.0 million must be funded on the Closing Date and expensed in 2003 and the remaining $10.0 million must be expensed prior to December 31, 2004; provided that notwithstanding the foregoing, an additional $10.0 million may be expensed in the aggregate following the Second Amendment Effective Date and prior to December 31, 2006.

11. Effectiveness. This Amendment shall become effective as of the date (the “Second Amendment Effective Date”) on which the following conditions are satisfied:

(a) The Administrative Agent shall have received counterparts hereof duly executed by HM, Holding, Intermediate Holding and the Administrative Agent;

(b) The Administrative Agent shall have received consent letters from the Requisite Lenders authorizing the Administrative Agent to enter into this Amendment;

(c) As of the Second Amendment Effective Date, the representations and warranties contained herein and in the other Credit Documents made by any Credit Party

6

shall hereby be deemed to be repeated on and as of the Second Amendment Effective Date and shall be true and correct in all material respects on and as of the Second Amendment Effective Date, except to the extent such representations and warranties specifically relate to an earlier date, in which case such representations and warranties shall have been true and correct in all material respects on and as of such earlier date;

(d) On or prior to the Second Amendment Effective Date, HM shall have permanently reduced the Revolving Commitments by $75,000,000 by delivering notice to the Administrative Agent as provided for by Section 2.13 of the Credit Agreement; and

(e) As of the Second Amendment Effective Date, no Default or Event of Default shall have occurred and be continuing or shall be caused hereby.

12. New Term Loan Intercreditor. Each Lender executing this Second Amendment hereby authorizes and instructs the Administrative Agent to enter into amendments and restatements or such other modifications of any Collateral Document as the Administrative Agent reasonably determines necessary in connection with and prior to the incurrence of the Indebtedness permitted pursuant to Section 6.1(y) in order to secure the New Term Loans on an equal and ratable basis with the Obligations, including without limitation by the amendment of the definition of “Majority Secured Parties” to include the holders of the New Term Loans on the same ratable basis as the existing “Secured Parties” in respect of the existing “Secured Obligations” which terms shall be likewise modified, or in place of such amendments, restatements or modifications to the Collateral Documents, to enter into the New Term Loan Intercreditor Agreement to achieve the same result in respect of any Liens granted to secure the New Term Loans by Grantors on Collateral (which shall be the only Liens granted in respect of the New Term Loans); provided that (i) any such modification shall provide that the New Term Loans are not provided any more benefits or rights than, and are subject to at least the same obligations as, the existing Secured Parties under the Collateral Documents and (ii) any such amended, restated or otherwise modified Collateral Document or New Term Loan Intercreditor Agreement shall be distributed to each Lender at least five Business Days prior to its effectiveness.

13. Continuing Effect of Credit Agreement. This Amendment shall not be construed as a waiver of or consent to any further or future action on the part of the Credit Parties that would require a waiver or consent by the Administrative Agent and/or the Lenders. Except as expressly amended hereby, the Credit Agreement shall continue to be and shall remain in full force and effect in accordance with its terms.

14. Counterparts. This Amendment may be executed by one or more of the parties hereto on any number of separate counterparts, and all of said counterparts taken together shall be deemed to constitute one and the same instrument. Delivery of an executed signature page of this Amendment by facsimile transmission shall be effective as delivery of a manually executed counterpart hereof.

15. GOVERNING LAW. THIS AMENDMENT SHALL BE GOVERNED BY, AND CONSTRUED AND INTERPRETED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK.

16. Expenses. HM agrees to pay or reimburse the Administrative Agent for all of its out-of-pocket costs and expenses incurred in connection with the preparation, negotiation

7

and execution of this Amendment, including, without limitation, the fees and disbursements of counsel to the Administrative Agent.

17. Miscellaneous. To facilitate reference to the provisions of the Credit Agreement, as amended by this Second Amendment, each Lender executing this Second Amendment hereby authorizes the Administrative Agent, on its behalf, to enter into an amendment and restatement of the Credit Agreement, at the Administrative Agent’s option, as amended by this Second Amendment, including, without limitation, so as to give effect to revisions to the Credit Agreement to conform to the New Term Loan Documentation, as expressly provided for pursuant to Section 6.1(y)(1) of the Credit Agreement; provided that any such amendment and restatement shall be distributed to each Lender.

8

IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be executed and delivered by their duly authorized officers as of the date first written above.

S-1

S-2

S-3

S-4

S-5

S-6

S-7

S-8

S-9

S-10

S-11

S-12

SCHEDULE A TO

SECOND AMENDMENT TO

AMENDED AND RESTATED

CREDIT AND GUARANTY AGREEMENT

NEW TERM LOANS

In all events, any New Term Loans incurred under Section 6.1(y) shall conform with the following:

(i) the aggregate annual scheduled amortization in respect of any Series of New Term Loans shall not exceed 5.0% of the principal amount of such Series prior to the Revolving Loan Maturity Date;

(ii) the applicable New Term Loan Maturity Date of each Series shall be no shorter than six months following the latest of the final maturity of the Revolving Loans; and

(iii) the rate of interest applicable to the New Term Loans of each Series shall be determined by Company and the applicable new Lenders and shall be set forth in the New Term Loan Documentation; provided however that the interest rate applicable to the New Term Loans shall not be greater than the highest interest rate that may, under any circumstances, be payable with respect to Revolving Loans plus 0.25% per annum unless the interest rate with respect to the Revolving Loans are increased so as to equal the interest rate applicable to the New Term Loans.

A-1