The organization began its existence in 1928 as a chapter of a national association of attorneys employed by the federal government. Because the national organization refused to admit African Americans, including Francis P. Giles, an Assistant United States Attorney from the Eastern District of New York, and a number of Assistants from the Southern District of New York, the group broke with the national organization. It was created as a separate organization, the Federal Bar Association of New York, New Jersey and Connecticut, by act of the New York State Legislature, signed into law by then Governor Franklin D. Roosevelt, effective April 1, 1932.
The organization sponsored the groundbreaking ceremonies for the Federal Courthouse at 40 Centre Street, Foley Square, on July 20, 1932, as well as the dinner at the Hotel Commodore upon the opening of the "new" courthouse in 1936. It was instrumental in lobbying for the creation of law clerks for federal district court judges. It participated in pressing for the adoption of the Federal Rules of Civil Procedure for which it was praised by Dean, later, Judge Charles E. Clark. Chief Justice Charles Evans Hughes was a frequent attendee at the organization's functions. In 1938, the organization adopted a resolution condemning the Rumanian government for endeavoring to oust Jewish members of the bar from their jobs.
The tradition of the Thanksgiving Luncheon dates back to at least 1943. The Law Day Dinner was instituted in 1962.
In 1968, the organization changed its name to the Federal Bar Council and made the courts of the Second Circuit its primary focus. The name "Council" was adopted because it reflected the advisory role to the courts which it had historically played, much like the Law Council of Inns of Court in England.
In 1969, the Federal Bar Council conducted its first Winter Bench & Bar Conference which has been held every year since. In 2000, the Council inaugurated a Fall Bench & Bar Retreat which in 2019 will be held at the Mohonk Mountain House, New Paltz, NY, from Friday, November 1, to Sunday, November 3, 2019. The 52nd Annual Winter Bench & Bar Conference will be held at the Rosewood Baha Mar, Nassau, Bahamas from Sunday, February 23, to Sunday, March 1, 2020.