Computational complexity: Difference between revisions

From Cornell University Computational Optimization Open Textbook - Optimization Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Authors: Steve Bentioulis, Augie Bravo, Will Murphy (SysEn 6800 Fall 2020)
Authors: Sarang Harshey (sh2669) (SysEn 5800 Fall 2024)
 
Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu


== Introduction ==
== Introduction ==
<blockquote>''“The subject of my talk is perhaps most directly indicated by simply asking two questions: first, is it harder to multiply than to add? and second, why?...there is no algorithm for multiplication computationally as simple as that for addition, and this proves something of a stumbling block” - Alan Cobham, 1965'' <ref>[https://www.cs.toronto.edu/~sacook/homepage/cobham_intrinsic.pdf A. Cobham, The intrinsic computational difficulty of functions], in Y. Bar-Hillel, ed., Logic, Methodology and Philosophy of Science: Proceedings of the 1964 International Congress, North-Holland Publishing Company, Amsterdam, 1965, p. 24-30 </ref></blockquote>
Computational complexity is a study of resources, especially time and space, required to solve a computational problem. It provides an understanding of how the resource requirement scales-up as the problem gets bigger and bigger. Another objective is to know which problems are solvable using algorithms and which problems are truly difficult and practically impossible to solve.  
Computational complexity refers to the amount of resources needed to solve a problem. Complexity increases as the amount of resources required increases. While this notion may seem straightforward enough, computational complexity has profound impacts.  
 
'''Complexity Classes'''
 
Problems can be grouped into sets that share a common property and require similar amount of computational resources. These sets or groups are known as ''Complexity Classes''. Some basic types of complexity classes are mentioned below.
 
'''P (Polynomial Time) Class''': If a problem can be solved in polynomial time, then it is considered as a Class P problem. Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input. Linear programming problems are considered under P class. [1]


The quote above from Alan Cobham is some of the earliest thinking on defining computational complexity and set the stage for defining problems based on complexity classes to indicate the feasibility of computational problems.
'''NP (Non-deterministic Polynomial Time) Class''': If the solution of a problem can be verified in polynomial time, then the problem is considered as a Class NP problem. These problems require a solution to be available which can be verified, however finding solution itself may take more than polynomial time. [1]


Additionally, the theory of computational complexity is in its infancy and has only been studied in earnest starting in the 20<sup>th</sup> century.  
'''NP-Hard Class''': NP-Hard problems may or may not fall under NP Class. A problem H is NP-Hard if for every problem L in NP, there exists a polynomial time reduction from L to H. These are usually optimization problems. [1]


== Theory, Methodology ==
'''NP-Complete Class''': NP-Complete problems are a subset of NP class. If all the problems in a NP can be reduced to problem H in polynomial time, then problem H is considered as NP-Complete. NP Complete problems are the hardest problems to solve. These are usually decision problems. [1]
The concept of computation has evolved since the advent of the standard universal electronic computer and the associated widespread societal adoption. And while the electronic computer is often synonymous with computation, it is important to remember that computation is a major scientific concept irrespective of whether it is conducted by machine, man, or otherwise.


When studying computation, a key area of interest is understanding what problems are, in fact, computable. Researchers have shown that some tasks are inherently incomputable in that no computer can solve them without going into infinite loops on certain inputs. This phenomenon begs the question how do you determine if a problem can be computed, moreover, for those problems that can be computed how can you calculate the complexity associated with computing the answer.
'''History'''


The focus of computational complexity is the measure of computational efficiency quantifying the amount of computational resources required to solve a problem.<ref>Arora, S., & Barak, B. (2009). Computational complexity: a modern approach. Cambridge: Cambridge University Press. Retrieved from https://cornell-library.skillport.com/skillportfe/main.action?path=summary/BOOKS/31235</ref>
The history of computational complexity can be traced back to the paper, ''“On computable numbers with an application to the Entsheidungs problem”,'' published in 1936 by A M Turing [2]. In this paper, Turing introduced his famous Turing Machine and provided a convincing formalization of computable function. In following years, once the theory was established to determine which problem can and cannot be solved, questions regarding relative computational difficulty came up [3]. This can be considered as the birth of computational complexity. Rabin M O in his paper, ''“Degree of difficulty of computing a function and partial ordering or recursive sets”,'' in 1960 made an attempt to measure the amount of work inherent in computing a function along with defining the degree of difficulty of computing a function [4]. Another influential paper that was published in 1965 by Hartmanis and Stearns, “''On the computational complexity of algorithms''”, discussed the method of categorizing problems according to how hard they are to compute. As part of this paper, a variety of theorems were established. It also provided a way to classify functions or recognition problems according to their computational complexity and laid out definitions of time complexity and space complexity [5]. Paper by Alan Cobham in 1965, ''“The intrinsic computational difficulty of a function”'', discussed the relative difficulty between addition and multiplication and characterised an important class of functions computable in time bounded by a polynomial in the length of input [6]. These papers from Cobhem, Hartemanis and Rabin can be considered as the founding papers for the field of computational complexity. Manuel Blum in 1967, formulated axioms known as Blum Axioms, that specify desirable properties of complexity measures on the set of computation function. Field of computational complexity started to grow from 1971 onwards, with studies from Steven Cook, Leonid Levin and Richard Karp which expanded the study of problems belonging to class NP-Complete.


Within the study of computational complexity there exists the notion of a complexity class defined as a set of functions that can be computed within given resource bounds.<ref>Du, D., & Ko, K.-I. (2014). Theory of computational complexity. (Second edition.). Hoboken, New Jersey: Wiley. Retrieved from http://onlinelibrary.wiley.com/book/10.1002/9781118595091</ref>
'''Need for Computational Complexity'''


=== Class P ===
Computational complexity provides a method to analyse an algorithm in terms of complexity and provides information on the performance that can be expected. In a complex algorithm, through computational complexity, costliest steps (in terms of space and time) can be identified and efforts can be made for improving efficiency by tuning these steps before implementation.  It also helps in selecting the best algorithms and eliminate inefficient algorithms.
Class P computational complexity problems refer to those that can solved in polynomial running time, where “P” stands for polynomial and running time is a function of the number of bits in the input.<ref>Arora, S., & Barak, B. (2009). Computational complexity: a modern approach. Cambridge: Cambridge University Press. Retrieved from https://cornell-library.skillport.com/skillportfe/main.action?path=summary/BOOKS/31235</ref>


A complexity class refers to a specific decision problem rather than generic types of problems. For example, it is not acceptable to state that integer multiplication is in class P. Rather you must state the specific decision problem, e.g. the product of 3 and 5 is a class P problem.
== Algorithm Discussion ==
Computational complexity can be calculated by algorithm analysis by establishing a relationship between length of the algorithm input and the number of steps it takes (time complexity) or amount of space it takes (space complexity). This relationship is usually represented as a function. The smaller the value of this function, the higher the efficiency of the algorithm. Function value also indicates the growth rate with respect to size of the input.


Furthermore, the running time is defined by minutes or nanoseconds, but refers to the number of operations to be performed to resolve or verify an answer to a problem.  Running time is a function of the number of bits being input into the decision problem. This allows us to ignore the efficiency of the machine running the computation and judge the complexity of the decision problem solely by the merits of such a problem.
Mathematical notations, usually known as ''Asymptotic Notations'', are used to describe the complexity of any algorithm. There are broadly three notations used to describe complexity, namely – Big-O (''O'' - Notation), Big-Theta (Θ - Notation) and Big-Omega (Ω - Notation).


=== Class NP ===
'''''O - notation''''' represents the upper bound or the worst case, i.e. maximum time (or space) required for an algorithm. For functions ''f'' and ''t,''
NP stands for nondeterministic polynomial time, originally referring to nondeterministic Turing machines (NDTM) in which the Turing machine has two transition functions and the computer arbitrarily determines which transition function to apply.


Complexity class NP consists of problems that can be efficiently verified within a running time upper bounded by polynomial function. Verifiability is the concept that if given a potential solution to the problem it can be confirmed or rejected.
''f(n) = O (t(n))''


==== Class NP-hard and NP-complete ====
if ''f(n) ≤ c.t(n) for'' all ''n ≥ d,''
The NP-hard complexity class is a subcategory of the NP complexity class that defines those problems that are at least as hard as any other language in NP. If P ≠ NP, then NP-hard problems cannot be decided in polynomial time. See P vs NP on this page.


NP-complete refers to those problems within the NP complexity-class that are the hardest problems to solve within the NP class. Examples of NP-complete problems include Independent Set, [https://optimization.cbe.cornell.edu/index.php?title=Traveling_salesman_problem Traveling Salesperson], Subset Sum, and Integer Programming problems. The implication of these problems is that they are not in P unless P = NP.
where ''c'' and ''d'' are positive integers ''[7] [8]''


=== P vs NP ===
'''''Ω''''' '''''- notation''''' represents lower bound or the best case, i.e. minimum time (or space) required for an algorithm. For functions, ''f'' and ''t,''
The difference between class P and class NP computational complexity is illustrated simply by considering a Sudoku puzzle. Ask yourself, is it easier to solve a Sudoku puzzle or verify whether an answer to a Sudoku puzzle is correct? Class P refers to computational complexity problems that can be efficiently solved. Class NP refers to those problems which have answers that can be efficiently verified. The answer to a Sudoku problem can be efficiently verified and for that reason is considered a class NP complexity problem.


This then begs the question that for every class NP problem, i.e. those that can be efficiently verified, does that mean they can also be efficiently solved? If so, then P = NP. However, we have not yet been able to prove that P = NP and thus the implications that P ≠ NP must also be considered.
''f(n) = Ω (t(n))''


The importance of understanding P vs NP is the subject of much discussion and has even sparked competition in the scientific community. The problem of P vs NP was selected by the Clay Mathematics Institute (CMI) of Cambridge, Massachusetts as one of seven most difficult and important problems to be solved at the dawn of the 21<sup>st</sup> century. A prize of $1 million has been allocated for anyone that can bring forward a solution.<ref>Clay Mathematics Institute, The Millennium Prize Problems. Retrieved from http://https://www.claymath.org/millennium-problems/millennium-prize-problems</ref>
if ''c.t(n) ≤ f(n)'' for all ''n ≥ d,''


=== Methodology ===
where ''c'' and ''d'' are positive integers ''[7] [8]''
The methodology for determining computational complexity is built upon the notion of a Turing machine and quantifying the number of computational operations the machine is to perform to resolve or verify a problem. A straight-forward approach would be to quantify the number of operations required considering every possible input to the Turing machine’s algorithm. This approach is referred to as worst-case complexity as it is the most possible number of operations to be performed in order to solve the problem.


However, critics of worst-case complexity highlight that in practice the worst-case behaviors of algorithms may never actually be encountered, and the worst-case approach can be unnecessarily cumbersome. As an alternative, average-case analysis seeks to design efficient algorithms that apply to most real-life instances. An important component of average-case analysis is the concept of an average-graph distribution of the inputs. There are several approaches to determining the average-graph including randomization. An average-case problem consists of both a decision problem and an average-graph distribution of inputs, implying that the complexity of a decision problem can vary with the inputs.<ref>Arora, S., & Barak, B. (2009). Computational complexity: a modern approach. Cambridge: Cambridge University Press. Retrieved from https://cornell-library.skillport.com/skillportfe/main.action?path=summary/BOOKS/31235</ref>  
'''''Θ - notation''''' represents average-case as it gives both upper and lower bounds of the algorithm. For functions, ''f'' and ''t,''
 
''f(n) = Θ (t(n))'',
 
if ''c<sub>1</sub>.t(n) ≤ f(n) ≤ c<sub>2</sub>.t(n)'' for all ''n ≥ d,''
 
where ''c<sub>1</sub>'', ''c<sub>2</sub>'' and ''d'' are positive integers '' [7] [8]''
 
'''Big-O (O-notation)''' is most commonly used to analyse the complexity of an algorithm as it provides the worst-case estimate. It considers only the number of steps involved for the problem resolution and not the hardware configuration that’s being used. Some of the common complexities are mentioned below.
 
* '''''Constant Time Complexity O(1)''''' : computation time for the problem is independent of size of the input data.
* '''''Linear Time Complexity O(n)''''' : computation time for the problem is proportional to the size of the input n and increase linearly.
* '''''Polynomial Time Complexity O(n<sup>c</sup>)''''' : computation time for the problem is polynomial times of size of the input ''n'' where ''c'' is a constant depending on the problem.
* '''''Exponential Time Complexity O(2<sup>n</sup>)''''' : computation time for the problem doubles with every addition of input ''n.''


== Numerical Example ==
== Numerical Example ==
The efficiency of a computation problem is measured by the operations executed to solve, not the seconds (or years) required to solve it. The number of operations executed is a function of input size and arrangement. The big O notation is used to determine an algorithm’s complexity class according to the number of operations it performs as a function of input.<ref>Mohr, A. Quantum Computing in Complexity Theory and Theory of Computation (PDF). Retrieved from http://www.austinmohr.com/Work_files/complexity.pdf</ref>
Complexity is defined based on the number of steps involved. Below are few examples demonstrating complexity calculation by analysing the algorithm.


The notation O(n) is used where ‘O’ refers to the order of a function and ‘n’ represents the size of the input.<ref>A. Mejia, How you can change the world by learning Data Structures and Algorithms. Retrieved from: https://towardsdatascience.com/how-you-can-change-the-world-by-learning-data-structures-and-algorithms-84566c1829e3</ref>
'''Example of Linear Time Complexity''' or '''''O(n)'':'''


An example of an O(1) problem includes determining whether one number is odd or even. The algorithm reads a bit of input and performs one operation to determine whether or not the number is odd or even. No matter how large or small the quantity of inputs the number of operations holds constant at 1; for that reason this is an O(1) problem.
* Let’s consider below algorithm with a “for” loop.
 
''x = 0''
 
''y = 0''
 
''a = 3''
 
''b = 3''
 
''for i in range (a):''
 
''x = x + 5''
 
''for i in range (b):''
 
''y = y + 10''
 
''print (x,y)''
 
* Algorithm Output: 15 30
* For the first loop, total 3 operations were performed, one each for a=1, a=2, a=3
* Similarly for the second loop, 3 more operations were performed one each for b=1, b=2, b=3
* Therefore, total 6 operations were performed
* Hence, time complexity for this algorithm is ''O(a+b)'' ⇒ ''O(n)'' where ''n'' is generic representation of the total input size
 
'''Example of Polynomial Time Complexity''' or '''''O(n<sup>k</sup>)'':'''
 
* Let’s consider an example of algorithm with a nested “''for''” loop
 
''x = 0''
 
''y = 0''
 
''a = 3''
 
''b = 3''
 
''for i in range (a):''
 
''for j in range (b):''
 
''x = x + j''
 
''print (x, end = “ “)''
 
''print ( )''
 
* Output of this algorithm would be:
 
1          3          6
 
7          9          12
 
13        15        18
 
* In this algorithm, for every iteration of outer loop, i.e. for each "a" (a=1, a=2, a=3), inner loop runs for "b" times (b=1, b=2, b=3)
 
* Therefore, a total of 9 operations were performed
 
* Hence, the complexity of this algorithm is ''O(a*b) ⇒ O(n<sup>2</sup>)'' where ''n'' is generic representation of input size


An example of O(n) problem includes identifying the minimum input within an unsorted array. To compute this problem the computer must read every bit of input to determine whether or not it is less than the prior bit of input. For this reason, the number of operations is linearly correlated to the quantity of inputs. For example, the decision problem of finding the minimum of {5,9,3,2,7,1,4} requires the computer to check every element in the array. This array has n=7 inputs, so it will require 7 operations to read each bit an determine if it is less than the prior bit. This scales linearly as the size of input increases.
== Applications ==
== Applications ==
Computational Complexity is influential to numerous scientific fields including [https://optimization.cbe.cornell.edu/index.php?title=Quantum_computing_for_optimization quantum computing], game theory, data mining, and cellular automata.<ref>Robert A. Meyers. (2012). Computational Complexity: Theory, Techniques, and Applications. Springer: Springer. Retrieved from https://search.ebscohost.com/login.aspx?custid=s9001366&groupid=main&profid=pfi&authtype=ip,guest&direct=true&db=edspub&AN=edp1880523&site=eds-live&scope=site</ref> Focusing in on quantum computing, there are important applications to the study of computational complexity as the theory of complexity is largely based upon the Turing machine and the Church-Turing thesis that any physically realizable computation device can be simulated by a Turing machine. If quantum computers are to be physically realizable they could alter our understanding of how complex a decision problem may be by providing enhanced methods in which algorithms may be computed and potentially lowering the number of operations to be performed.<ref> Arora, S., & Barak, B. (2009). Computational complexity: a modern approach. Cambridge: Cambridge University Press. Retrieved from https://cornell-library.skillport.com/skillportfe/main.action?path=summary/BOOKS/31235 </ref>
Computational complexity helps developers to determine the most efficient algorithm that consumes least number of resources (time and space). It is widely used in various fields. Some of the major application areas are discussed below.
 
'''Computer science and algorithm efficiency analysis''': It helps in designing efficient algorithms and aids in software/ hardware selection. It enables comparison of multiple algorithms that can be used for solving a specific problem. Computational complexity helps in optimizing database queries and information retrieval processes.
 
'''Quantum Computing''': Quantum computing is majorly used for cybersecurity, artificial intelligence, data management, data analytics and optimization. With the help of quantum complexity theory, quantum algorithms can be made exponentially efficient compared to classical algorithms. This enables businesses to solve problems which cannot be solved by conventional computing devices.
 
'''Game Theory:''' Game theory deals with optimal decision making in any strategic setting. It is extensively used for analysing customer behaviour, market trend, develop effective business strategies, competitive product pricing etc. Computational complexity helps in defining the most efficient algorithm for approaching the game and to come up with best outcome while considering all the possible variations.
 
'''Healthcare system:''' Various aspects of healthcare system such as patient management, continuity of patient care, nursing etc requires efficient decision-making processes. These complex tasks are broken down into fairly smaller and manageable tasks using computational models while giving due consideration to interrelatedness of systems. '''[9]'''


== Conclusion ==
== Conclusion ==
Computational complexity has important implications in the field of computer science and far reaching applications that span numerous fields and industries. As computable problems become more complex the ability to increase the efficiency in which they are solved becomes more important. Advancements toward solving P vs NP will have far reaching impacts on how we approach the computability of problems and the ability to efficiently allocate resources.
Computational complexity provides a framework for analysing, comparing and selecting the most efficient algorithm and also enables the selection of best suited hardware and software combination. The algorithms or problems are categorised in classes P, NP, NP-Complete, NP-Hard and the worst-case complexity is represented using Big-O notation. Computational complexity has been evolving continuously for past few decades and remains a major area of study. Computational complexity is extensively used in the field of game theory, quantum mechanics, health system where the number of computing is huge and cannot be solved using conventional methods.


== Sources ==
== Sources ==
<references />
<references />

Latest revision as of 13:32, 10 December 2024

Authors: Sarang Harshey (sh2669) (SysEn 5800 Fall 2024)

Stewards: Nathan Preuss, Wei-Han Chen, Tianqi Xiao, Guoqing Hu

Introduction

Computational complexity is a study of resources, especially time and space, required to solve a computational problem. It provides an understanding of how the resource requirement scales-up as the problem gets bigger and bigger. Another objective is to know which problems are solvable using algorithms and which problems are truly difficult and practically impossible to solve.

Complexity Classes

Problems can be grouped into sets that share a common property and require similar amount of computational resources. These sets or groups are known as Complexity Classes. Some basic types of complexity classes are mentioned below.

P (Polynomial Time) Class: If a problem can be solved in polynomial time, then it is considered as a Class P problem. Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input. Linear programming problems are considered under P class. [1]

NP (Non-deterministic Polynomial Time) Class: If the solution of a problem can be verified in polynomial time, then the problem is considered as a Class NP problem. These problems require a solution to be available which can be verified, however finding solution itself may take more than polynomial time. [1]

NP-Hard Class: NP-Hard problems may or may not fall under NP Class. A problem H is NP-Hard if for every problem L in NP, there exists a polynomial time reduction from L to H. These are usually optimization problems. [1]

NP-Complete Class: NP-Complete problems are a subset of NP class. If all the problems in a NP can be reduced to problem H in polynomial time, then problem H is considered as NP-Complete. NP Complete problems are the hardest problems to solve. These are usually decision problems. [1]

History

The history of computational complexity can be traced back to the paper, “On computable numbers with an application to the Entsheidungs problem”, published in 1936 by A M Turing [2]. In this paper, Turing introduced his famous Turing Machine and provided a convincing formalization of computable function. In following years, once the theory was established to determine which problem can and cannot be solved, questions regarding relative computational difficulty came up [3]. This can be considered as the birth of computational complexity. Rabin M O in his paper, “Degree of difficulty of computing a function and partial ordering or recursive sets”, in 1960 made an attempt to measure the amount of work inherent in computing a function along with defining the degree of difficulty of computing a function [4]. Another influential paper that was published in 1965 by Hartmanis and Stearns, “On the computational complexity of algorithms”, discussed the method of categorizing problems according to how hard they are to compute. As part of this paper, a variety of theorems were established. It also provided a way to classify functions or recognition problems according to their computational complexity and laid out definitions of time complexity and space complexity [5]. Paper by Alan Cobham in 1965, “The intrinsic computational difficulty of a function”, discussed the relative difficulty between addition and multiplication and characterised an important class of functions computable in time bounded by a polynomial in the length of input [6]. These papers from Cobhem, Hartemanis and Rabin can be considered as the founding papers for the field of computational complexity. Manuel Blum in 1967, formulated axioms known as Blum Axioms, that specify desirable properties of complexity measures on the set of computation function. Field of computational complexity started to grow from 1971 onwards, with studies from Steven Cook, Leonid Levin and Richard Karp which expanded the study of problems belonging to class NP-Complete.

Need for Computational Complexity

Computational complexity provides a method to analyse an algorithm in terms of complexity and provides information on the performance that can be expected. In a complex algorithm, through computational complexity, costliest steps (in terms of space and time) can be identified and efforts can be made for improving efficiency by tuning these steps before implementation.  It also helps in selecting the best algorithms and eliminate inefficient algorithms.

Algorithm Discussion

Computational complexity can be calculated by algorithm analysis by establishing a relationship between length of the algorithm input and the number of steps it takes (time complexity) or amount of space it takes (space complexity). This relationship is usually represented as a function. The smaller the value of this function, the higher the efficiency of the algorithm. Function value also indicates the growth rate with respect to size of the input.

Mathematical notations, usually known as Asymptotic Notations, are used to describe the complexity of any algorithm. There are broadly three notations used to describe complexity, namely – Big-O (O - Notation), Big-Theta (Θ - Notation) and Big-Omega (Ω - Notation).

O - notation represents the upper bound or the worst case, i.e. maximum time (or space) required for an algorithm. For functions f and t,

f(n) = O (t(n))

if f(n) ≤ c.t(n) for all n ≥ d,

where c and d are positive integers [7] [8]

Ω - notation represents lower bound or the best case, i.e. minimum time (or space) required for an algorithm. For functions, f and t,

f(n) = Ω (t(n))

if c.t(n) ≤ f(n) for all n ≥ d,

where c and d are positive integers [7] [8]

Θ - notation represents average-case as it gives both upper and lower bounds of the algorithm. For functions, f and t,

f(n) = Θ (t(n)),

if c1.t(n) ≤ f(n) ≤ c2.t(n) for all n ≥ d,

where c1, c2 and d are positive integers  [7] [8]

Big-O (O-notation) is most commonly used to analyse the complexity of an algorithm as it provides the worst-case estimate. It considers only the number of steps involved for the problem resolution and not the hardware configuration that’s being used. Some of the common complexities are mentioned below.

  • Constant Time Complexity O(1) : computation time for the problem is independent of size of the input data.
  • Linear Time Complexity O(n) : computation time for the problem is proportional to the size of the input n and increase linearly.
  • Polynomial Time Complexity O(nc) : computation time for the problem is polynomial times of size of the input n where c is a constant depending on the problem.
  • Exponential Time Complexity O(2n) : computation time for the problem doubles with every addition of input n.

Numerical Example

Complexity is defined based on the number of steps involved. Below are few examples demonstrating complexity calculation by analysing the algorithm.

Example of Linear Time Complexity or O(n):

  • Let’s consider below algorithm with a “for” loop.

x = 0

y = 0

a = 3

b = 3

for i in range (a):

x = x + 5

for i in range (b):

y = y + 10

print (x,y)

  • Algorithm Output: 15 30
  • For the first loop, total 3 operations were performed, one each for a=1, a=2, a=3
  • Similarly for the second loop, 3 more operations were performed one each for b=1, b=2, b=3
  • Therefore, total 6 operations were performed
  • Hence, time complexity for this algorithm is O(a+b)O(n) where n is generic representation of the total input size

Example of Polynomial Time Complexity or O(nk):

  • Let’s consider an example of algorithm with a nested “for” loop

x = 0

y = 0

a = 3

b = 3

for i in range (a):

for j in range (b):

x = x + j

print (x, end = “ “)

print ( )

  • Output of this algorithm would be:

1          3          6

7          9          12

13        15        18

  • In this algorithm, for every iteration of outer loop, i.e. for each "a" (a=1, a=2, a=3), inner loop runs for "b" times (b=1, b=2, b=3)
  • Therefore, a total of 9 operations were performed
  • Hence, the complexity of this algorithm is O(a*b) ⇒ O(n2) where n is generic representation of input size

Applications

Computational complexity helps developers to determine the most efficient algorithm that consumes least number of resources (time and space). It is widely used in various fields. Some of the major application areas are discussed below.

Computer science and algorithm efficiency analysis: It helps in designing efficient algorithms and aids in software/ hardware selection. It enables comparison of multiple algorithms that can be used for solving a specific problem. Computational complexity helps in optimizing database queries and information retrieval processes.

Quantum Computing: Quantum computing is majorly used for cybersecurity, artificial intelligence, data management, data analytics and optimization. With the help of quantum complexity theory, quantum algorithms can be made exponentially efficient compared to classical algorithms. This enables businesses to solve problems which cannot be solved by conventional computing devices.

Game Theory: Game theory deals with optimal decision making in any strategic setting. It is extensively used for analysing customer behaviour, market trend, develop effective business strategies, competitive product pricing etc. Computational complexity helps in defining the most efficient algorithm for approaching the game and to come up with best outcome while considering all the possible variations.

Healthcare system: Various aspects of healthcare system such as patient management, continuity of patient care, nursing etc requires efficient decision-making processes. These complex tasks are broken down into fairly smaller and manageable tasks using computational models while giving due consideration to interrelatedness of systems. [9]

Conclusion

Computational complexity provides a framework for analysing, comparing and selecting the most efficient algorithm and also enables the selection of best suited hardware and software combination. The algorithms or problems are categorised in classes P, NP, NP-Complete, NP-Hard and the worst-case complexity is represented using Big-O notation. Computational complexity has been evolving continuously for past few decades and remains a major area of study. Computational complexity is extensively used in the field of game theory, quantum mechanics, health system where the number of computing is huge and cannot be solved using conventional methods.

Sources