Abstract
•The feasibility of firefly algorithm (FFA) for intra mode decision is investigated.•A novel objective function for FFA is proposed to perform intra mode decision.•The convergence of FFA’s fireflies is considered as the optimal stopping point.
High Efficiency Video Coding (HEVC) is the latest encoder that increased the intra modes from 9 to 35 to efficiently handle the contents of the video. The HEVC’s test model (HM) selects the optimal intra mode using the brute force method which increases the complexity of HEVC. This work, firstly, investigates the feasibility of firefly algorithm (FFA) due to its exploration and exploitation characteristics to expedite the intra mode decision in HEVC. Secondly, a novel objective function is formulated for FFA to efficiently compute the brightness for intra modes in FFA. Thirdly, the parameters of FFA are made dynamic to adjust according to the contents of video sequences. Simulation results demonstrate that the nature inspired algorithm, FFA, pays off by saving a minimum of 27% of the total coding time on average and doesn’t sacrifice quality by limiting Bjontegaard delta bit rate (BD-BR) increase to only 0.98% on average.