Both served only one term as president. While the frequency of two-term presidents has varied depending on the era, it is worth pointing out that Adams and Carter served during times in which most presidents won a second term. Of the first five presidents before Adams, only one, his father, John Adams, failed to win reelection. Similarly, four of the five presidents that have served after Carter have served two terms, with George H.W. Bush being the sole exception. That Adams and Carter both failed to win reelection at times when it was common for presidents to serve for eight years seems to invite further examination of their presidential and post-presidential careers.
Perhaps in large part because of the characteristics listed above, both have not been highly regarded by historians, at least in terms of their presidencies. Their post-presidential careers are a different story and again, there are similarities.
Both were highly intellectual and this trait did not come across well to many Americans. In their bids for reelection, the intellectualism of Adams and Carter became liabilities. Anti-intellectualism was a dominant feature of American life in the 1830s and it still is today (perhaps even more so). Think of Fox News viewers, for example. Millions of Americans lack a basic scientific literacy and express considerable doubt on a number of issues where the informed, academic, and scholarly communities are nearly unanimous, including climate change, evolution, and vaccinations.
Both Adams and Carter were highly religious men who used their religion to support progressive causes, rather than conservative ones. I am defining "conservative" here to mean traditional; support for the status quo in terms of hierarchical social relations and economic inequality; militarism; nationalism; patriarchy, etc. Adams took on the Slave Power by opposing the annexation of Texas and Gag Rule. Carter has developed a long record of standing up for human rights, international diplomacy, and safeguards against unfettered capitalism and the unabashed accumulation of material wealth.
Both had successful political careers after serving as president. This is exceedingly rare since most presidents tend to serve relatively late in life and many have died in office. Other than William Howard Taft, who went on to serve as chief justice of the Supreme Court, it is hard to think of any other politicians besides Adams and Carter who have accomplished more after occupying the White House. Adams went on to serve in the House of Representatives (to my knowledge, the only ex-president to do so).
Both embraced scientific knowledge as a means to improve society. Adams advocated a system of weights and measures while Carter installed solar panels on the White House lawn. Adams's belief in "improvement" was not merely religious; it was economic, too. Years ahead of his time, Carter warned us of the dangers of consuming fossil fuels and urged us to turn down our heaters and conserve gas in our cars.
Both lived to a very old age. Adams lived until he was eighty years old. Considering he was born in the colonial era, even before there was a United States of America, this was remarkable. Carter, as of this writing, is still alive, going strong into his nineties!
Both were relatively independent in terms of party loyalty, going against party stalwarts when they saw fit. Adams refused to appoint spoilsmen to public office. Carter bucked his party on many issues, including financial deregulation and dam-building in the West, in spite of large Democratic majorities in Congress.
The similarities between Adams and Carter are amplified further by examining the men who succeeded them. Both were followed by presidents who contained a lot of similarities; that of Jackson and Reagan, respectively. Although Jackson was a Democrat and Reagan a Republican, in many ways, they served similar constituencies and promoted similar political ideologies. This is why a number of people have written about how today's Democrats are distancing themselves from what was once the party of Jefferson and Jackson. Many statewide Democratic Parties are no longer holding Jefferson-Jackson dinners. If Jefferson and Jackson were alive today, they'd probably be closer to the modern Republican Party, though this is not as clear-cut as one might think and making such broad comparisons over long periods of time can be problematic.
But back to Jackson and Reagan...notice my deliberate use of the word "successful." I might have chosen the word "good," but I didn't. Aside from the fact that "good" is highly relative, subjective, and imprecise, my point here is that a successful president is not the same as a good president. James K. Polk was successful in achieving practically all of his stated goals: acquiring California, acquiring Oregon, lowering the tariff, etc. But was this good for the country? A lot of people back then didn't think so and even more today are critical of the fact that the United States, on highly dubious grounds, launched an imperialistic assault on Mexico with the eventual prize of acquiring one-third to one-half of the latter's northern territories. Reagan, similarly, was a successful president who set the nation on a trajectory from which it has barely moved, at least in terms of demonizing the federal government, inflaming racial resentment among the white working-class at the expense of African Americans and other minorities, deriding welfare recipients, deregulating the financial sector, decimating unions, unleashing corporate corruption, empowering the military, and caving to Christian fundamentalism.
Jackson and Reagan were beloved by their admirers but have recently come under much deserved scrutiny by historians. Both left a legacy that lasted well beyond their deaths. We often think of the era between approximately 1824 and 1850 as the "Jacksonian era," though the historian Louis Masur has argued persuasively that this label gives Jackson too much credit. Both appointed people to public office who were ideologically rigid (think of Anton Scalia) and who continued to serve well after their appointers had passed along. Think of the damage that Roger B. Taney and Scalia have done in terms of constitutional jurisprudence. Both appropriated anti-intellectual ideas and skepticism of academic expertise to great advantage. Both spoke for the white working-class and unsurprisingly, neither Jackson nor Reagan were very kind to African Americans. Both ostensibly believed in "limited government" but departed from this ideology when necessary or politically expedient. Both made strong appeals to nationalism and patriotism. Both had members of their own party question their credentials and intellect. Thomas Jefferson purportedly was unimpressed with Jackson's oratory even though Jackson conceived of himself as an old-school Jeffersonian Republican. Richard Nixon and Barry Goldwater were not impressed with Reagan and did not think it would be a good thing if he became president. The list of similarities continues...
In closing, Reagan may have been the Andrew Jackson of his time and Jimmy Carter may have been the John Quincy Adams of his time.