Motivated by delay‐sensitive information transmission applications, we solve an optimal power allocation problem with a K‐block delay constraint on data transmission using a cooperative relay network assuming a block fading channel model. Channel information is fed back to the transmitter only in a causal fashion, so that the optimal power allocation strategy is only based on the current and past channel gains. We consider the two simplest schemes for information transmission using a three node (a source, a relay and a destination) relay network, namely the amplify and forward (AF) and decode and forward (DF) protocols. We use a dynamic programming (DP) based methodology to solve a (K‐block delay constrained) general expected cost optimisation problem with a short term (over K blocks) sum power (total transmission power of the source and the relay) constraint. By specialising the cost function appropriately, we solve the delay constrained outage minimisation problem in this paper. We also propose a simple but sub‐optimal power allocation scheme based on a high signal to noise ratio (SNR) approximation, which is computationally much less demanding than the DP‐based optimal method. Extensive numerical results are presented for Rayleigh and Rician fading channels, including results demonstrating the performance gain obtained by optimally allocating the (sum of source and relay) power to the different blocks as opposed to equally distributing the total power across all blocks. The accuracy of the high SNR approximation based power allocation scheme is also illustrated.