A perusal of The Helsinki Principles on the Law of Maritime Neutrality, a basis for international law on the subject, shows that Israel was well within its prescriptions in forcibly stopping the Gaza-bound flotilla.
The relevant clauses in the Helsinki Principles read as follows:
5.1.1 Neutral ships in belligerent ports
A neutral ship in a belligerent port enjoys the same protection against attacks as civilian objects in land warfare… Neutral warships in belligerent ports retain their right of self-defense.
5.1.2 Protection against attacks
(3) Merchant ships flying the flag of a neutral State may be attacked if they are believed on reasonable grounds to be carrying contraband or breaching a blockade, and after prior warning they intentionally and clearly refuse to stop, or intentionally and clearly resist visit, search, capture or diversion.
(4) Merchant ships flying the flag of a neutral State may be attacked if they
(a) engage in belligerent acts on behalf of the enemy;
(b) act as auxiliaries to the enemy’s armed forces;
(c) are incorporated into or assist the enemy’s intelligence system;
(d) sail under convoy of enemy warships or military aircraft; or
(e) otherwise make an effective contribution to the enemy’s military action, e.g., by carrying military materials, and it is not feasible for the attacking forces to first place passengers and crew in a place of safety. Unless circumstances do not permit, they are to be given a warning, so that they can re-route, off-load, or take other precautions.
5.2.1 Visit and search
… [B]elligerent warships have a right to visit and search vis-à-vis neutral commercial ships in order to ascertain the character and destination of their cargo. If a ship tries to evade this control or offers resistance, measures of coercion necessary to exercise this right are permissible. This includes the right to divert a ship where visit and search at the place where the ship is encountered are not practical.
5.2.10 Blockade
Blockade, i.e. the interdiction of all or certain maritime traffic coming from or going to a port or coast of a belligerent, is a legitimate method of naval warfare. In order to be valid, the blockade must be declared, notified to belligerent and neutral States, effective and applied impartially to ships of all States. A blockade may not bar access to neutral ports or coasts. Neutral vessels believed on reasonable and probable grounds to be breaching a blockade may be stopped and captured. If they, after prior warning, clearly resist capture, they may be attacked.