Butler, Benjamin Franklin, 1818-1893

Benjamin Franklin Butler was born in Deerfield, New Hampshire, the sixth and youngest child of John Butler and Charlotte Ellison Butler. His father served under General Andrew Jackson at the Battle of New Orleans during the War of 1812 and later became a privateer, dying of yellow fever in the West Indies not long after Benjamin was born. He was named after Founding Father Benjamin Franklin. His elder brother, Andrew Jackson Butler (1815–1864), would serve as a colonel in the Union Army during the Civil War and joined him in New Orleans. Butler's mother was a devout Baptist who encouraged him to read the Bible and prepare for the ministry. In 1827, at the age of nine, Butler was awarded a scholarship to Phillips Exeter Academy, where he spent one term. He was described by a schoolmate as "a reckless, impetuous, headstrong, boy", and regularly got into fights.

Butler's mother moved the family in 1828 to Lowell, Massachusetts, where she operated a boarding house for workers at the textile mills. He attended the public schools there, from which he was almost expelled for fighting, the principal describing him as a boy who "might be led, but could not be driven." He attended Waterville (now Colby) College in pursuit of his mother's wish that he prepare for the ministry, but eventually rebelled against the idea. In 1836, Butler sought permission to go instead to West Point for a military education, but did not receive one of the few places available. He continued his studies at Waterville, where he sharpened his rhetorical skills in theological discussions, and began to adopt Democratic political views. He graduated in August 1838. Butler returned to Lowell, where he clerked and read law as an apprentice with a local lawyer. He was admitted to the Massachusetts bar in 1840, and opened a practice in Lowell.

After an extended courtship, Butler married Sarah Hildreth, a stage actress and daughter of Dr. Israel Hildreth of Lowell, on May 16, 1844. They had four children: Paul (1845–1850), Blanche (1847–1939), Paul (1852–1918) and Ben-Israel (1855–1881). Butler's business partners included Sarah's brother Fisher, and her brother-in-law, W. P. Webster.

In 1844, Butler was elected as a member to the American Philosophical Society.

Butler quickly gained a reputation as a dogged criminal defense lawyer who seized on every misstep of his opposition to gain victories for his clients, and also became a specialist in bankruptcy law. His trial work was so successful that it received regular press coverage, and he was able to expand his practice into Boston.

Butler's success as a lawyer enabled him to purchase shares in Lowell's Middlesex Mill Company when they were cheap. Although he generally represented workers in legal actions, he also sometimes represented mill owners. This adoption of both sides of an issue manifested when he became more politically active. He first attracted general attention by advocating the passage of a law establishing a ten-hour day for laborers, but he also opposed labor strikes over the matter. He instituted a ten-hour work day at the Middlesex Mills.

During the debates over the ten-hour day a Whig-supporting Lowell newspaper published a verse suggesting that Butler's father had been hanged for piracy. Butler sued the paper's editor and publisher for that and other allegations that had been printed about himself. The editor was convicted and fined $50, but the publisher was acquitted on a technicality. Butler blamed the Whig judge, Ebenezer Rockwood Hoar, for the acquittal, inaugurating a feud between the two that would last for decades and significantly color Butler's reputation in the state.

Butler, as a Democrat, supported the Compromise of 1850 and regularly spoke out against the abolition of slavery. However, at the state level, he supported the coalition of Democrats and Free Soilers that elected George S. Boutwell governor in 1851. This garnered him enough support to win election to the state legislature in 1852. His support for Franklin Pierce as president, however, cost him the seat the next year. He was elected a delegate to the 1853 state constitutional convention with strong Catholic support, and was elected to the state senate in 1858, a year dominated by Republican victories in the state. Butler was nominated for governor in 1859 and ran on a pro-slavery, pro-tariff platform; he narrowly lost to incumbent Republican Nathaniel Prentice Banks.

In the 1860 Democratic National Convention at Charleston, South Carolina, Butler initially supported John C. Breckinridge for president, but then shifted his support to Jefferson Davis, believing that only a moderate Southerner could keep the Democratic party from dividing. A conversation he had with Davis prior to the convention convinced him that Davis might be such a man, and he gave him his support before the convention split over slavery. Butler ended up supporting Breckinridge over Douglas against state party instructions, ruining his standing with the state party apparatus. He was nominated for governor in the 1860 election by a Breckinridge splinter of the state party, but trailed far behind other candidates.

Although he sympathized with the South, Butler stated "I was always a friend of southern rights but an enemy of southern wrongs" and sought to serve in the Union Army. His military career before the Civil War began as a private in the Lowell militia in 1840. Butler eventually rose to become colonel of a regiment of primarily Irish American men. In 1855, the nativist Know Nothing Governor Henry J. Gardner disbanded Butler's militia, but Butler was elected brigadier general after the militia was reorganized. In 1857 Secretary of War Jefferson Davis appointed him to the Board of Visitors of West Point. These positions did not give him any significant military experience.

After Abraham Lincoln was elected president in November 1860, Butler traveled to Washington, D.C. When a secessionist South Carolina delegation arrived there he recommended to lameduck President James Buchanan that they be arrested and charged with treason. Buchanan refused the idea. Butler also met with Jefferson Davis and learned that he was not the Union man that Butler had previously thought he was. Butler then returned to Massachusetts, where he warned Governor John A. Andrew that hostilities were likely and that the state militia should be readied. He took advantage of the mobilization to secure a contract with the state for his mill to supply heavy cloth to the militia. Military contracts would constitute a significant source of profits for Butler's mill throughout the war.

Butler also worked to secure a leadership position should the militia be deployed. He first offered his services to Governor Andrew in March 1861. When the call for militia finally arrived in April, Massachusetts was asked for only three regiments, but Butler managed to have the request expanded to include a brigadier general. He telegraphed Secretary of War Simon Cameron, with whom he was acquainted, suggesting that Cameron issue a request for a brigadier and general staff from Massachusetts, which soon afterward appeared on Governor Andrew's desk. He then used banking contacts to ensure that loans that would be needed to fund the militia operations would be conditioned on his appointment. Despite Andrew's desire to assign the brigadier position to Ebenezer Peirce, the bank insisted on Butler, and he was sent south to ensure the security of transportation routes to Washington. The nation's capital was threatened with isolation from free states because it was unclear whether Maryland, a slave state, would also secede.

The two regiments Massachusetts sent to Maryland were the 6th and 8th Volunteer Militia. The 6th departed first and was caught up in a secessionist riot in Baltimore, Maryland on April 19. Butler traveled with the 8th, which left Philadelphia the next day amid news that railroad connections around Baltimore were being severed. Butler and the 8th traveled by rail and ferry to Maryland's capital, Annapolis, where Governor Thomas H. Hicks attempted to dissuade them from landing. Butler landed his troops (which needed food and water), occupying the Naval Academy. When Hicks informed Butler that no one would sell provisions to his force, Butler pointed out that armed men did not necessarily have to pay for needed provisions, and he would use all measures necessary to ensure order.

After being joined by the 7th New York Militia, Butler directed his men to restore rail service between Annapolis and Washington via Annapolis Junction, which was accomplished by April 27. He also threatened Maryland legislators with arrest if they voted in favor of secession and eventually seized the Great Seal of Maryland. Butler's prompt actions in securing Annapolis were received with approval by the US Army's top general, Winfield Scott, and he was given formal orders to maintain the security of the transit links in Maryland. In early May, Scott ordered Butler to lead the operations that occupied Baltimore. On May 13 he entered Baltimore on a train with 1000 men and artillery, with no opposition. That was done in contravention to Butler's orders from Scott, which had been to organize four columns to approach the city by land and sea. General Scott criticized Butler for his strategy (despite its success) as well as his heavy-handed assumption of control of much of the civil government, and he recalled him to Washington. Butler shortly after received one of the early appointments as major general of the volunteer forces. His exploits in Maryland also brought nationwide press attention, including significant negative press in the South, which concocted stories about him that were conflations of biographical details involving not just Butler but also a namesake from New York and others. When two Massachusetts regiments had been sent overland to Maryland, two more were dispatched by sea under Butler's command to secure Fort Monroe at the mouth of the James River. After being dressed down by Scott for overstepping his authority, Butler was next assigned command of Fort Monroe and of the Department of Virginia. On May 27, Butler sent a force 8 miles (13 km) north to occupy the lightly defended adjacent town of Newport News, Virginia at Newport News Point, an excellent anchorage for the Union Navy. The force established and significantly fortified Camp Butler and a battery at Newport News Point that could cover the entrance to the James River ship canal and the mouth of the Nansemond River. Butler also expanded Camp Hamilton, established in the adjacent town of Hampton, Virginia, just beyond the confines of the fort and within the range of its guns.

The Union occupation of Fort Monroe was considered a potential threat on Richmond by Confederate General Robert E. Lee, and he began organizing the defense of the Virginia Peninsula in response. Confederate General John B. Magruder, seeking to buy time while awaiting men and supplies, established well-defended forward outposts near Big and Little Bethel, only 8 miles (13 km) from Butler's camp at Newport News as a lure to draw his opponent into a premature action. Butler took the bait, and suffered an embarrassing defeat in the Battle of Big Bethel on June 10. Butler devised a plan for a night march and operation against the positions but chose not to lead the force in person for which he was later criticized. The plan proved too complex for his inadequately trained subordinates and troops to carry out, especially at night, and was further marred by the failure of staff to communicate all passwords and precautions. A friendly fire incident during the night gave away the Union position, which was further harmed the advance without knowledge of the layout or the strength of the Confederate positions. Massachusetts militia general Ebenezer Peirce, who commanded in the field, received the most criticism for the failed operation. With the withdrawal of many of his men for use elsewhere, Butler was unable to maintain the camp at Hampton although his forces retained the camp at Newport News. Butler's commission, which required approval from Congress, was vigorously debated after Big Bethel, with critical comment raised about his lack of military experience. His commission was narrowly approved on July 21, the day of the First Battle of Bull Run, the war's first large-scale battle. The battle's poor Union outcome was used as cover by General Scott to reduce Butler's force to one incapable of substantive offense, and it was implicit in Scott's orders that the troops were needed nearer to Washington.

In August, Butler commanded an expeditionary force that, in conjunction with the United States Navy, took Forts Hatteras and Clark in North Carolina. That move, the first significant Union victory after First Bull Run, was lauded in Washington and won Butler accolades from President Lincoln. Butler was thereafter sent back to Massachusetts to raise new forces. That thrust Butler into a power struggle with Governor Andrew, who insisted on maintaining his authority to appoint regimental officers, refusing to commission (among others) Butler's brother Andrew and several of the general's close associates. The spat instigated a recruiting war between Butler and the state militia organization. The dispute delayed Butler's return to Virginia, but he was in November instead assigned to command of ground troops for operations in Louisiana.

While in command at Fort Monroe, Butler declined to return to their owners fugitive slaves who had come within his lines. He argued that Virginians considered them to be chattel property, and that they could not appeal to the Fugitive Slave Law of 1850 because of Virginia's secession. Furthermore, slaves used as laborers for building fortifications and other military activities could be considered contraband of war. It was later made standard Union Army policy to not return fugitive slaves. This policy was soon extended to the Union Navy.

Butler directed the first Union expedition to Ship Island, off the Mississippi Gulf Coast, in December 1861, and in May 1862 commanded the force that conducted the capture of New Orleans after its occupation by the Navy following the Battle of Forts Jackson and St. Philip. In the administration of that city he showed great firmness and political subtlety. He devised a plan for relief of the poor, demanded oaths of allegiance from anyone who sought any privilege from government, and confiscated weapons.

However, Butler's subtlety seemed to fail him as the military governor of New Orleans when it came to dealing with its Jewish population, about which the general, referring to local smugglers, infamously wrote, in October 1862: "They are Jews who betrayed their Savior, & also have betrayed us." Butler was considered "notorious for his anti-Semitism."

In an ordinary year, it was not unusual for as much as 10 percent of the city's population to die of yellow fever. In preparation, Butler imposed strict quarantines and introduced a rigid program of garbage disposal. As a result, in 1862, only two cases were reported.

Many of his acts, however, were highly unpopular. Most notorious was Butler's General Order No. 28 of May 15, 1862, that if any woman should insult or show contempt for any officer or soldier of the United States, she shall be regarded and shall be held liable to be treated as a "woman of the town plying her avocation," i.e., a prostitute. This was in response to various and widespread acts of overt verbal and physical abuse from the women of New Orleans, including cursing at and spitting on Union soldiers and pouring out chamber pots on their heads from upstairs windows when they passed in the street (with Admiral David Farragut being perhaps the most notable victim of a chamberpot attack).

There was no overt sexual connotation in Butler's order, but its effect was to revoke the protected status held by women under the social mores of the time, which mandated that any "respectable" woman (i.e., a non-prostitute) be treated with the extra degree of respect due a lady, regardless of her own provocations. Under General Order 28, however, if a woman showed any form of insult or contempt towards a Union soldier (even so much as turning her back when he approached or refusing to answer his questions), the usual social standards no longer applied, and she could be retaliated against (either verbally or physically) as if she were a common prostitute. The order produced the desired effect, as few women proved willing to risk retaliation simply to protest the Union presence; but it was seen as extremely draconian by everyone except the Union soldiers in New Orleans, and provoked general outrage in the South, as well as abroad, particularly in England and France.

He was nicknamed "Beast Butler" or alternatively "Spoons Butler," the latter nickname deriving primarily from an incident in which Butler seized a 38-piece set of silverware from a New Orleans woman attempting to cross the Union lines. Although the woman's pass permitted her to carry nothing but clothing on her person (making her carriage of the silverware illegal), the single set of silverware would have normally been considered protected personal valuables, and Butler's insistence on prosecuting the woman as a smuggler and seizing the silverware as wartime contraband under his dictate of confiscating all property of those "aiding the Confederacy" provoked angry jeers from white residents of New Orleans and the much-repeated perception that he used his power to engage in the petty looting of the household valuables of New Orleanians.

Shortly after the Confiscation Act of 1862 became effective in September, Butler increasingly relied upon it as a means of grabbing cotton. Since the Act permitted confiscation of property owned by anyone "aiding the Confederacy," Butler reversed his earlier policy of encouraging trade by refusing to confiscate cotton brought into New Orleans for sale. Firstly, he conducted a census in which 4,000 respondents failing to pledge loyalty to the Union were banished. Their property was seized and sold at low auction prices in which Andrew was often the prime buyer. Next, the general sent expeditions into the countryside with no military purpose other than to confiscate cotton from residents who were assumed to be disloyal. Once brought into New Orleans, the cotton would be similarly sold in rigged auctions. To maintain correct appearances, auction proceeds were dutifully held for the benefit of "just claimants", but the Butler consortium still ended up owning the cotton at bargain prices. Always inventive of new terminology to achieve his ends, Butler sequestered, or made vulnerable to confiscation, such "properties" in all of Louisiana beyond parishes surrounding New Orleans.

Butler censored New Orleans newspapers. When William Seymour, the editor of the New-Orleans Commercial Bulletin, asked Butler what would happen if the newspaper ignored his censorship, an angry Butler reportedly stated, "I am the military governor of this state — the supreme power — you cannot disregard my order, Sir. By God, he that sins against me, sins against the Holy Ghost." When Seymour published a favorable obituary of his father, who had been killed serving in the Confederate army in Virginia, Butler confiscated the newspaper and imprisoned Seymour for three months.

Butler closed The Picayune when it ran an editorial that he found offensive. Historian John D. Winters wrote that most of the newspapers "were allowed to reopen later but were so rigidly controlled that all color and interest were drained away" and that churches that planned a special day of prayer and fasting for the Confederacy were forbidden from doing so. Several clergymen were placed under arrest for refusing to pray for President Lincoln. The Episcopal churches were closed, and their three ministers were sent to New York City under military escort.

On June 7, 1862, Butler ordered the execution of William B. Mumford for tearing down a United States flag placed by Admiral Farragut on the United States Mint in New Orleans. In his memoirs, Butler maintained that Mumford had assembled a party of men, torn down the flag, dragged it through the streets and then trampled and spat on it and then ripped it into pieces and that Mumford distributed the remnants among members of the party who wore it as if it were a badge of honor, all of which was against the laws of war. Before Mumford was executed, Butler permitted him to make a speech for as long as he wished, and Mumford defended his actions by claiming that he was acting out of a high sense of patriotism. Most, including Mumford and his family, expected Butler to pardon him. The general refused to do so, but promised to care for his family if necessary. (After the war, Butler fulfilled his promise by paying off a mortgage on Mumford's widow's house and helping her find government employment.) For the execution and General Order No. 28, he was denounced (December 1862) by Confederate President Jefferson Davis in General Order 111 as a felon deserving capital punishment, who, if captured, should be reserved for execution.

Butler also took aim at foreign consuls in New Orleans. He ordered the seizure of $800,000 that had been deposited in the office of the Dutch consul, imprisoned the French champagne magnate Charles Heidsieck, and took particular aim at George Coppell of Great Britain, whom he suspended for refusal to cooperate with the Union. Instead, Butler accused Coppell of giving aid to the Confederate cause.

U.S. Secretary of State William Henry Seward sent Reverdy Johnson to New Orleans to investigate complaints of foreign consuls against certain Butler policies. Even when told by President Lincoln to restore a sugar shipment claimed by Europeans, Butler undermined the order. He also imposed a strict quarantine to protect against yellow fever, which had the added impact of delaying foreign commerce and bringing complaints to his headquarters from most foreign consuls.

With the Union occupation, runaway slaves and slaves from abandoned plantations arrived in large numbers in New Orleans. The unattached people had to be fed and housed. A Union officer complained of "a big problem" with the new arrivals. John D. Winters wrote, "Soldiers resented the fact that the pampered Negro was given better tents, equal rations, and was allowed to tear down more fences for sleeping boards than were the soldiers. General Phelps [an abolitionist] had organized a few squads of Negroes and drilled them daily.... Not knowing what to do with so many Negroes, Butler at first returned the runaway slaves to their masters. But still the contrabands came. Some of them were employed as cooks, nurses, washwomen, and laborers.... [Finally] Butler ordered... the exclusion of all unemployed Negroes and whites from his lines."

Although Butler's governance of New Orleans was popular in the North, where it was seen as a successful stand against recalcitrant secessionists, some of his actions, notably those against the foreign consuls, concerned Lincoln, who authorized his recall in December 1862. Butler was replaced by Nathaniel P. Banks. The necessity of taking sometimes radical actions and the support he received in Radical Republican circles drove Butler to change political allegiance, and he joined the Republican Party. He also sought revenge against the more moderate Secretary of State Seward, whom he believed to be responsible for his eventual recall.

Butler continues to be a disliked and controversial figure in New Orleans and the rest of the South.

Butler's popularity with the radicals meant that Lincoln could not readily deny him a new posting. Lincoln considered sending him to position in the Mississippi River area in early 1863, and categorically refused to send him back to New Orleans. He finally gave Butler command of the Department of Virginia and North Carolina in November 1863, based in Norfolk, Virginia. In January 1864, Butler played a pivotal role in the creation of six regiments of U.S. Volunteers recruited from among Confederate prisoners of war ("Galvanized Yankees") for duty on the western frontier. In May, the forces under his command were designated the Army of the James. On November 4, 1864, while in command of the Army of the James, Butler arrived in New York City with 3,500 troops. Secretary of War Edwin Stanton had ordered the troops there to prevent disorder on Election Day, November 8, and because of fear of Confederates coming from Canada to burn the city on Election Day. ″Even though he knew nothing about the plot [to burn the city] and did nothing to prevent it, Butler's mere presence with his 3,500 troops″ demoralized the leaders of the conspiracy, who postponed it until November 25, when it failed.

On September 27, 1862, Butler formed the first African-American regiment in the US Army, the 1st Louisiana Native Guard, and commissioned 30 officers to command it at the company level. This was highly unusual, as most USCT regiments were commanded by white officers only. "Better soldiers never shouldered a musket," Butler wrote, "I observed a very remarkable trait about them. They learned to handle arms and to march more easily than intelligent white men. My drillmaster could teach a regiment of Negroes that much of the art of war sooner than he could have taught the same number of students from Harvard or Yale." The regiment would serve Butler effectively during the Siege of Port Hudson.

General Butler also commanded a number of United States Colored Troops regiments which he deployed in combat during the Battle of Chaffin's Farm (sometimes also called the Battle of New Market Heights). The troops performed extremely well, and in the case of the 38th United States Colored Troops regiment, who had overcome overwhelming fire, heavy casualties and thick physical obstacles to overwhelm a more powerful force, he awarded a number of men the Medal of Honor. He also ordered a special medal designed and struck and awarded to 200 African-American soldiers who had served with distinction in the engagement. This was later called the Butler Medal.

Ulysses S. Grant, who did not think highly of Butler's military skills, ordered him to attack in the direction of Petersburg from the east, destroying the rail links supplying Richmond and distracting Robert E. Lee, in conjunction with attacks Grant would make from the north. Although Petersburg at this time was lightly defended and Butler could have occupied it with little difficulty, he hesitated and allowed a greatly inferior Confederate force under General Pierre G.T. Beauregard to box up the Army of the James on the Bermuda Hundred Peninsula. As a result, the Army of Northern Virginia arrived and dug in around Petersburg, resulting in an eight-month siege of the city. However, it was his mismanagement of the expedition against Fort Fisher, North Carolina, that finally led to his recall by General Grant. Butler devised a scheme to sail a boat filled with gunpowder up to the fort and detonate it, breaching its defenses, after which infantry would land ashore and storm the place. The plan went completely awry when the boat exploded prematurely in the harbor outside Ft. Fisher, doing no damage whatsoever and was barely even noticed by the Confederate troops manning the fort. Butler landed his infantry ashore, then gave up, recalled them, and reported back that Ft. Fisher was impossible to capture. Afterwards, Admiral David Dixon Porter informed Grant that it could be taken easily if anyone competent were put in charge.

Although Grant had largely been successful in removing incompetent political generals from service, Butler proved to be one such officer that could not be easily gotten rid of. As a prominent Radical Republican, Butler was also under consideration as a possible opponent of Lincoln in that year's election, and Lincoln had asked Butler to serve as his vice president in early 1864. After the election, however, Grant wrote to Secretary of War Edwin M. Stanton in early 1865 asking free rein to relieve Butler from military service. Since Stanton was traveling outside Washington, D.C., at the time, Grant appealed directly to Lincoln for permission to terminate Butler, noting "there is a lack of confidence felt in [Butler's] military ability". In General Order Number 1, Lincoln relieved Butler from command of the Department of North Carolina and Virginia and ordered him to report to Lowell, Massachusetts.

Grant informed Butler of his recall on January 8, 1865, and named Major General Edward O. C. Ord to replace him as commander of the Army of the James. Rather than report to Lowell, Butler went to Washington, where he used his considerable political connections to get a hearing before the Joint Congressional Committee on the Conduct of the War in mid-January. At his hearing Butler focused his defense on his actions at Fort Fisher. He produced charts and duplicates of reports by subordinates to prove he had been right to call off his attack of Fort Fisher, despite orders from General Grant to the contrary. Butler claimed the fort was impregnable. To his embarrassment, a follow-up expedition led by Maj. Gen. Alfred H. Terry and Brig. Gen. Adelbert Ames (Butler's future son-in-law) captured the fort on January 15, and news of this victory arrived during the committee hearing; Butler's military career was over. He was formally retained until November 1865 with the idea that he might act as military prosecutor of Confederate President Jefferson Davis.

Negative perceptions of Butler were compounded by his questionable financial dealings in several of his commands, as well as the activities of his brother Andrew, who acted as Butler's financial proxy and was given "almost free rein" to engage in exploitative business deals and other "questionable activities" in New Orleans. Upon arriving in the city, Butler immediately began attempts to participate in the lucrative inter-belligerent trade. He used a Federal warship to send $60,000 in sugar to Boston where he expected to sell it for $160,000. However, his use of the government ship was reported to the military authorities, and Butler was chastised. Instead of earning a profit, military authorities permitted him to recover only his $60,000 plus expenses. Thereafter, his brother Andrew officially represented the family in such activities. Everyone in New Orleans believed that Andrew accumulated a profit of $1–$2 million while in Louisiana. Upon inquiry from Treasury Secretary Chase in October 1862, the general responded that his brother actually cleared less than $200,000. When Butler was replaced in New Orleans by Major General Nathaniel Banks, Andrew Butler unsuccessfully tried to bribe Banks with $100,000 if Banks would permit Andrew's "commercial program" to be carried out "as previous to [Banks's] arrival."

Butler's administration of the Norfolk district was also tainted by financial scandal and cross-lines business dealings. Historian Ludwell Johnson concluded that during that period: "... there can be no doubt that a very extensive trade with the Confederacy was carried on in [Butler's Norfolk] Department.... This trade was extremely profitable for Northern merchants ... and was a significant help to the Confederacy.... It was conducted with Butler's help and a considerable part of it was in the hands of his relatives and supporters."

Shortly after arriving in Norfolk, Butler became surrounded by such men. Foremost among them was Brigadier General George Shepley, who had been military governor of Louisiana. Butler invited Shepley to join him and "take care of Norfolk." After his arrival, Shepley was empowered to issue military permits allowing goods to be transported through the lines. He designated subordinate George Johnston to manage the task. In fall 1864, Johnston was charged with corruption. However, instead of being prosecuted, he was allowed to resign after saying he could show "that General Butler was a partner in all [the controversial] transactions," along with the general's brother-in-law Fisher Hildreth. Shortly thereafter, Johnston managed a thriving between-the-lines trade depot in eastern North Carolina. There is no doubt that Butler was aware of Shepley's trading activities. His own chief of staff complained about them and spoke of businessmen who "owned" Shepley. Butler took no action.

Much of the Butler-managed Norfolk trade was via the Dismal Swamp Canal to six northeastern counties in North Carolina separated from the rest of the state by Albemarle Sound and the Chowan River. Although cotton was not a major crop, area farmers purchased bales from the Confederate government and took them through the lines where they would be traded for "family supplies." Generally, the Southerners returned with salt, sugar, cash, and miscellaneous supplies. They used the salt to preserve butchered pork, which they sold to the Confederate commissary. After Atlantic-blockaded ports such as Charleston and Wilmington were captured, this route supplied about ten thousand pounds of bacon, sugar, coffee, and codfish daily to Lee's army. Ironically, Grant was trying to cut off Lee's supplies from the Confederacy when Lee's provender was almost entirely furnished from Yankee sources through Butler-controlled Norfolk. Grant wrote of the issue, "Whilst the army was holding Lee in Richmond and Petersburg, I found ... [Lee] ... was receiving supplies, either through the inefficiency or permission of [an] officer selected by General Butler ... from Norfolk through the Albemarle and Chesapeake Canal."

Butler's replacement, Major General George H. Gordon, was appalled at the nature of the ongoing trade. Reports were circulating that $100,000 of goods daily left Norfolk for Rebel armies. Grant instructed Gordon to investigate the prior trading practices at Norfolk, after which Gordon released a sixty-page indictment of Butler and his cohorts. It concluded that Butler associates, such as Hildreth and Shepley, were responsible for supplies from Butler's district pouring "directly into the departments of the Rebel Commissary and Quartermaster." Some Butler associates sold permits for cross-line trafficking for a fee. Gordon's report received little publicity, because of the end of the war and Lincoln's assassination.

At the urging of his wife, Butler actively sought another political position in the Lincoln administration, but this effort came to an end with Lincoln's assassination in April 1865. In March 1866, Butler argued in the U.S. Supreme Court on behalf of the United States in Ex parte Milligan, in which the Court held, against the United States, that military commission trials could not replace civilian trials when courts were open and where there was no war. Butler then turned his eyes to Congress and was elected in 1866 on a platform of civil rights and opposition to President Andrew Johnson's weak Reconstruction policies. He supported a variety of populist or social reform positions, including women's suffrage, an eight-hour workday for federal employees, and the issuance of greenback currency.

Butler served four terms (1867–75) before losing reelection, and was then once again elected in 1876 for a single term. As a former Democrat, he was initially opposed by the state Republican establishment, which was particularly unhappy with his support of women's suffrage and greenbacks. The more conservative party organization closed ranks against him to deny two attempts (in 1871 and 1873) to gain the Republican nomination for Governor of Massachusetts. In 1874, hostile Republicans led by Ebenezer Rockwood Hoar succeeded in denying him renomination for his Congressional seat.

In 1868, Butler was selected to be one of the managers of the impeachment of President Johnson before the Senate. Although Thaddeus Stevens was the principal guiding force behind the impeachment effort, he was aging and ill at the time, and Butler stepped in to become the main organizing force in the prosecution. The case was focused primarily on Johnson's removal of Secretary of War Edwin Stanton in violation of the Tenure of Office Act, and was weak because the constitutionality of the law had not been decided. The trial was a somewhat uncomfortable affair, in part because the weather was hot and humid, and the chamber was packed. The prosecution's case was a humdrum recitation of facts already widely known, and it was attacked by the defense's William Evarts, who drowned the proceedings by repeatedly objecting to Butler's questions, often necessitating a vote by the Senate on whether or not to allow the question. Johnson's defense focused on the point that his removal of Stanton fell within the bounds of the Tenure of Office Act. Despite some missteps by the defense, and Butler's vigorous cross-examination of defense witnesses, the impeachment failed by a single vote. In the interval between the trial and the Senate vote, Butler searched without success for substantive evidence that Johnson operatives were working to bribe undecided Senators. After acquittal on the first article voted on, Senate Republicans voted to adjourn for ten days, seeking time to possibly change the outcome on the remaining articles. During this time, Butler established a House committee to investigate the possibility that four of the seven Republican Senators who voted for acquittal had been improperly influenced in their votes. He uncovered some evidence that promises of patronage had been made and that money may have changed hands but was unable to decisively link these actions to any specific Senator.

Butler wrote the initial version of the Civil Rights Act of 1871 (also known as the Ku Klux Klan Act). After his bill was defeated, Representative Samuel Shellabarger of Ohio drafted another bill, only slightly less sweeping than Butler's, that successfully passed both houses and became law upon Grant's signature on April 20. Along with Republican Senator Charles Sumner, Butler proposed the Civil Rights Act of 1875, a seminal and far-reaching law banning racial discrimination in public accommodations. The Supreme Court of the United States declared the law unconstitutional in the 1883 Civil Rights Cases.

Butler managed to rehabilitate his relationship with Ulysses Grant after the latter became president, to the point where he was seen as generally speaking for the president in the House. He annoyed Massachusetts old-guard Republicans by convincing Grant to nominate one of his protégés to be collector of the Port of Boston, an important patronage position, and secured an exception for an ally, John B. Sanborn, in legislation regulating the use of contractors by the Internal Revenue Service for the collection of tax debts. In 1874, Sanborn would be involved in the Sanborn Contract scandal, in which he was paid over $200,000 for collecting debts that would likely have been paid without his intervention.

Butler greatly expanded his business interests during and after the Civil War, and was extremely wealthy when he died, with an estimated net worth of $7 million ($200 million today). Historian Chester Hearn believed "The source of his fortune has remained a mystery, but much of it came from New Orleans...." However, Butler's mills in Lowell, which produced woolen goods and were not hampered by cotton shortages, were economically successful during the war, supplying clothing and blankets to the Union Army, and regularly paying high dividends. Successful postwar investments included a granite company on Cape Ann and a barge freight operation on the Merrimack River. After learning that no domestic manufacturer produced bunting, he invested in another Lowell mill to produce it, and convinced the federal government to enact legislation requiring domestic sources for material used on government buildings. Less successful ventures included investments in real estate in the Virginia, Colorado, and the Baja Peninsula of western Mexico, and a fraudulent gold mining operation in North Carolina. He also founded the Wamesit Power Company and the United States Cartridge Company, and was one of several high-profile investors who were deceived by Philip Arnold in the famous Diamond hoax of 1872.

Butler put some of his money into more charitable enterprises. He purchased confiscated farms in the Norfolk, Virginia area during the war and turned them over to cooperative ventures managed by local African Americans, and sponsored a scholarship for African-Americans at Phillips Andover Academy. He also served for fifteen years in executive positions of the National Home for Disabled Volunteer Soldiers.

His law firm also expanded significantly after the war, adding offices in New York City and Washington. High-profile cases he took included the representation of Admiral David Farragut in his quest to be paid by the government for prizes taken by the Navy during the war, and the defense of former Secretary of War Simon Cameron against an attempted extortion in a salacious case that gained much public notice.

Butler built a mansion immediately across the street from the United States Capitol in 1873–1874, known as the Butler Building. One unit of the building was constructed to be fireproof so that it could be rented as storage for valuable and irreplaceable survey records, maps, and engraving plates of the U.S. Coast and Geodetic Survey, whose headquarters in the Richards Building was directly next door. The building was used by President Chester A. Arthur while the White House was being refurnished. On April 10, 1891, the Department of the Treasury purchased the building from Butler for $275,000, and it became the headquarters of the U.S. Marine Hospital Service, with its Hygienic Laboratory (the predecessor of the National Institutes of Health) occupying its top floor.

Butler ran unsuccessfully for Governor of Massachusetts in 1878 as an independent with Greenback Party support, and also sought the Democratic nomination. The latter was denied him by the party leadership, which refused to admit him into the party, but he was nominated by a populist rump group of Democrats who disrupted the main convention, forcing it to adjourn to another location. He was renominated in similar fashion in 1879; in both years, the Republicans won against the divided Democrats. Because Butler sought the governorship in part as a stepping stone to the presidency, he opted not to run again until 1882. In 1882, he was elected by a 14,000 margin after winning nomination by both Greenbacks and an undivided Democratic party.

As governor, Butler was active in promoting reform and competence in administration, in spite of a hostile Republican legislature and Governor's Council. He appointed the state's first Irish-American and African-American George Lewis Ruffin judges, and appointed the first woman to executive office, Clara Barton, to head the Massachusetts Reformatory for Women. He also graphically exposed the mismanagement of the state's Tewksbury Almshouse under a succession of Republican governors. Butler was somewhat notoriously snubbed by Harvard University, which traditionally granted honorary degrees to the state's governors. Butler's honorarium was denied because the Board of Overseers, headed by Ebenezer Hoar, voted against it.

Butler's bid for reelection in 1883 was one of the most contentious campaigns of his career. His presidential ambitions were well known, and the state's Republican establishment, led by Ebenezer and George Frisbie Hoar, poured money into the campaign against him. Running against Congressman George D. Robinson (whose campaign manager was a young Henry Cabot Lodge), Butler was defeated by 10,000 votes, out of more than 300,000 cast. Butler is credited with beginning the tradition of the "lone walk", the ceremonial exit from the office of Governor of Massachusetts, after finishing his term in 1884.

In 1882, Butler successfully litigated Juilliard v. Greenman before the Supreme Court. In what was seen as a victory for Greenback supporters, the case confirmed that the government had the right to issue paper currency for public and private debts. Butler leveraged the win to run for president in 1884. Nominated by the Greenback and Anti-Monopoly parties, he was unsuccessful in getting the Democratic nomination, which went to Grover Cleveland. Cleveland refused to adopt parts of Butler's platform in exchange for his political support, prompting Butler to run in the general election. He sought to gain electoral votes by engaging in fusion efforts with Democrats in some states and Republicans in others, in which he took what were perceived in the contemporary press as bribes $25,000 from the campaign of Republican James G. Blaine. The effort was in vain: Butler polled 175,000 out of 10 million cast.

In his later years Butler reduced his activity level, working on his memoir, Butler's Book, which was published in 1892, and serving from 1866 to 1879 as president of the National Home for Disabled Volunteer Soldiers.

Butler died on January 11, 1893 of complications from a bronchial infection a day after arguing a case before the Supreme Court. He is buried in his wife's family cemetery, behind the main Hildreth Cemetery in Lowell. The inscription on Butler's monument reads, "the true touchstone of civil liberty is not that all men are equal but that every man has the right to be the equal of every other man—if he can."

His daughter Blanche married Adelbert Ames, a Mississippi governor and senator who had served as a general in the Union Army during the war. Butler's descendants include the famous scientist Adelbert Ames, Jr., suffragist and artist Blanche Ames Ames, Butler Ames, Hope Butler, and George Plimpton.

According to biographer Hans L. Trefousse:

Butler was one of the most controversial 19th-century American politicians. Demagogue, speculator, military bungler, and sharp legal practitioner--he was all of these; and he also was a fearless advocate of justice for the downtrodden, a resourceful military administrator, and an astonishing innovator. He was passionately hated and equally strongly admired, and if the South called him "Beast," his constituents in Massachusetts were fascinated by him....As a leading advocate of radical Reconstruction, Butler played an important role in the conflict between president and Congress. His effectiveness was marred by the frequency with which engaged in personal altercations, and his conduct as one of the principal managers of the impeachment trial of Andrew Johnson was dubious. Nevertheless he deserves recognition as a persistent critic of southern terrorism and is one of the chief authors of the Civil Rights Act of 1875.