These compilations of presidential papers are official publications in themselves.
The American State Papers contain legislative and executive documents of Congress, 1789-1838. They include the messages and speeches of Washington, Adams, Jefferson and Madison, 1789-1814.
The U.S. Congressional Serial Set includes the texts of messages to Congress, including inaugural addresses and vetoes.
Find out more about these resources and how to access them in the section of the GIMMS Guide to the United States Congress about Committee Work.
Compiled by James Daniel Richardson and originally published in 10 volumes by GPO, 1896-99; subsequently updated. Available in the following formats:
"Each Public Papers volume contains the papers and speeches of the President of the United States that were issued by the Office of the Press Secretary during the specified time period. The material is presented in chronological order, and the dates shown in the headings are the dates of the documents or events." -- Archives.gov
Includes verbatim texts of speeches, news conferences, radio addresses, executive orders, etc., chronologically arranged.
Most of these documents are also available in the compilations listed above.
Executive orders have to do with the management of government business and have the force of law; E.O. 9066, for example, authorized the exclusion of people from "military areas," leading to the internment of Japanese-Americans during World War II.
Proclamations are frequently used to establish the National Week or Month of ..., but may also have more substantive content. Proclamation 6641 , for example, implemented NAFTA and Proclamation 2541  denied 8 accused German-American Nazi saboteurs access to civilian courts. (The Military Order establishing the Military Commission which tried the accused saboteurs is included in the Executive Orders.)
Since 1936 executive orders and proclamations have been published in the Federal Register and codified, along with other documents signed by the President, in the Code of Federal Regulations, Title 3: The President.
Here are some other places you can find Executive Orders and Proclamtions.
Inaugural speeches are traditionally given before Congress at the swearing-in ceremony at the beginning of a term and often express the president's intentions for his term.
State of the Union speeches are a constitutional requirement that intends to keep the Congress informed of current budget and economic issues and to permit the President to make legislative recommendations.
Farewell Addresses are informal good-byes delivered as a president leaves office. Not every president made one; Washington and Jackson's were not speeches at all, but published in newspapers.
"Veto" is the term used to indicate a president's unwillingness to sign a bill passed by Congress into law. There are two ways to veto a bill. First, the president may, with ten days (excluding Sundays) send the bill back to Congress with a list of reasons why he or she won't sign it. The Congress then has the opportunity to override the President's objection or the bill dies. The second way to veto a bill is to simply ignore it for those ten days. If the Congress adjourns during those ten days, the bill automatically dies. This is called a "pocket" veto. On the other hand, if the Congress does not adjourn during those ten days, then the bill is automatically ratified.