I am unsure of how to set this problem up. Can anyone help me?

Write an equation for the function that expresses the area of the printed portion in terms of the length and width of the poster. Use the derivative of that function to find the inflection points. Finally, study the inflection points to find the extreme values.

Welcome to Fluther. We’re not allowed to tell you exactly how to do a homework problem, but as it seems you only wanted us to point you in the right direction, your question is fine.

Call the length of your printed area L and the width W. So the printed area, which is the figure you’re trying to optimize, is A = L*W. The other information available to you is that the total area of your paper is 240 sq in, and you have margins of 1 in on the sides and bottom and 2 in on the top. How do you represent this information in a formula? Consider this: the width of your paper has a total of two extra inches added to W (1 in on the left side and 1 in on the right) and the length of the paper has three extra inches added to L (1 on bottom and 2 on top). The product of these dimensions must equal 240. Now you can set up a formula that represents this, and using these formulas and the steps given by @ratboy, solve the problem.

Let me know if you need more help.