A 3.5 brisket is a tough smoke to get right. Any little cuts like that are hard. There just isn't enough mass to make it cook evenly. You'll find that a larger brisket will be much easier to smoke. Do you know if it was a brisket flat or point? That can make a difference, too. I suspect it was a flat, if it was tough. The point has more internal fat, and usually cooks more tender.
Next time, try to find a larger brisket (the leftovers freeze well), may experiment with brining or injecting, and watch that internal temp. IT is everything. Time means nothing. If you saw 181 around 4-5 hours, it was probably in a stall. Next time, take it 195.