FOURTH AMENDMENT TO SHAREHOLDERS AGREEMENT

Exhibit 10.10.4

FOURTH AMENDMENT TO SHAREHOLDERS’ AGREEMENT

THIS AMENDMENT (the “Amendment”) to that certain Shareholders’ Agreement (defined below) is made between Fleetwood Homes Inc. (“Fleetwood”), Cavco Industries, Inc. (“Cavco”), and Third Avenue Trust, a Delaware business trust, on behalf of Third Avenue Value Fund (“TAVF” and together with Fleetwood and Cavco, the “Parties”) as of June 5, 2012. Subject to this Amendment, the terms and conditions of the Shareholders’ Agreement are incorporated herein by reference.

WHEREAS the Parties entered into that certain Shareholders’ Agreement dated as of August 17, 2009 (the “Shareholders’ Agreement”);

WHEREAS the Parties entered into a first amendment to the Shareholders’ Agreement dated as of November 30, 2010 (the “First Amendment”); and

WHEREAS the Parties entered into a second amendment to the Shareholders’ Agreement dated as of June 17, 2011 (the “Second Amendment”); and

WHEREAS the Parties entered into a third amendment to the Shareholders’ Agreement dated as of February 16, 2012 (the “Third Amendment”); and

WHEREAS the Parties now desire to amend the Shareholders’ Agreement.

NOW, THEREFORE, the Parties agree as follows:

The following sentence shall be added to the end of Section 7.1:

The gain on bargain purchase for the purchase of Palm Harbor Homes, Inc. ($22,009,000) shall be deducted from net income in determining whether net income exceeded $10 million for the fiscal year ended March 31, 2012. Therefore, for the purpose of determining a Triggering Period, net income did not exceed $10 million for the fiscal year ended March 31, 2012.

Provided and notwithstanding any other provision herein, the term “EBITDA” shall exclude the gain on bargain purchase for the purchase of Palm Harbor Homes, Inc. ($22,009,000) for the fiscal year ended March 31, 2012.

The undersigned, pursuant to due authority, have caused this Amendment to be executed as of the date set forth above.