2024 Cornell Optimization Open Textbook Feedback

Computational complexity

• "Problems in this class are efficiently solvable, however, the time required to solve grows polynomially with the size of input" Please revise the use of however in the rest of the file.
• Put the citation number within the punctuation. Please revise this problem for the rest of the file.
• Please pay attention to use proper punctuations. For example, add commas before and behind the cited paper title will improve the clarity of the sentence. Please also revise this issue for the rest of the file.
• "The smaller the value of this function, the higher the efficiency of the algorithm. "
• The example 1 has some grammar issue.
• This is topic-specific suggestion:
• Please try to come up with one example of computational complexity and give the procedure of how to reduce its complexity while retaining the solution quality.
• When stating potential applications, please provide related references.
• The provided numerical is somewhat trivial. You may try a more sophisticated example (e.g., comparing the different sorting algorithms).

Heuristic algorithms

• More details on the Application Section.
• Please come up with one numerical example with hill-climbing.
• Please add more references to support the content in the Introduction section.
• Please add figure captions for the pseudocodes.
• Avoid using contraction (e.g., we're) and pronouns (e.g., we) in scientific writing.
• ResearchGate is not a publisher. Please check the reference again.

Local branching

• If the "Local Branching " is preferedly used in the text, please revise the other typo like "local branching".
• Please explain every variable/parameters used in the equations.
• It may be good to generate a table or figure to summarize the results of the numerical examples.
• Page 5: "Local Branching is a powerful optimization strategy when it comes to solving MILP optimization problems. Its main applications lie in industries such as logistics, manufacturing, energy, data science and engineering for problems such as scheduling, production planning, nesting and transportation logistics where decision-makers look for high-quality solutions within reasonable computational times. " This sentence looks a little strange. Please revise it.
• It would be better to provide more references to support the statements in the Application Section.
• Please pay attention to the citation format. A link only may not be sufficient. Please refer to our examples for improve it.
• Please try to come up with flowchart/pseudocode for demonstrating the steps.
• Please replace Step 1, 2, 3 with correct subheadings in Application section.
• Websites and Youtube often are not proper citation sources.

Trust-region methods

• For the applications section, a better method is to briefly summarize the advantages of the method in multiple areas, instead of presents its application on two very specific cases.
• Please try to update equations on page 5.
• Please try to use the pseudocode for the procedure described on page 3.
• m_k and p in eq.(1) are not defined.
• Fix the typo “prediced reduction” in eq.(3).
• Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, us) in scientific writing.
• Rewrite the sentence, “Identical to line search, we do not need to compute the algorithm (2)” as (2) itself is not an algorithm.
• The termination conditions are not complete in section “Termination Criteria”.
• Fletcher is not the same author as cited in [5].

Quadratic programming

• "Quadratic programming problems typically are formatted as minimization problems, and the general mathematical formulation is "
• "For convex problems, Q defined in the equation above must be positive semi-definite; if not, there may be multiple local solutions meeting minimization criteria and deemed non-convex."
• "Active set methods are best suited for most linear programming problems, particularly those with manageable dimensions, as they exploit the problem's structure and update estimates of active constraints iteratively. "
• it would be better to provide definitions of all parameters/variables used in the pseudocode.
• same pseudocode problem
• "Constraints that are box defined or inequalities that are convex, are critical for the gradient projection method to be executed efficiently. Note, the box defined constraints in this instance will define our feasibility region and referred to as the box" This text is a little strange to read, please revise it.
• Insert picture as equation is ok for now, but be sure to use the equation form when do it on the Wiki page.
• In Wiki editing, step-by-step calculation process is not necessary. It may be better to keep the brevity of the solution process.
• "Markowitz reviewed a return projection of a given stock based on historical data, then analyzed variances from historical projections based on realized returns to generate the risk"
• "This problem has proven to be beneficial in optimizing the cargo loads and establishing transportation routes.
• "An optimized power management program is critical for the plug-in hybrid electric vehicles (PHEV) to operate efficiently. The power management problem is more complex in PHEVs than traditional hybrid vehicles and purely electric vehicles as it relies on grid charged batteries as the for its initial range, then drives like HEVs alternating between fuel and charge obtained during the combustion of the fuel. The power management problem can be described as a quadratic polynomial and solved with quadratic programing with the optimization goal to minimize fuel consumption.15" Please check the grammar of the texts above.
• "GAMS Gurobi suite contains several algorithms that are suitable for quadratic programming"
• check the grammer of the conclusion texts.
• Please provide one example of non convex QP problem.

Subgradient optimization

• "Developed in the Soviet Union during the 1960s and 70s, primarily by Naum Z. Shor" a citation may be needed here.
• "While similar in approach as the gradient methods for differentiable functions, there are several key differences. " This sentence reads a little bit strange. Please check the grammer for the entire file.
• Please revise the use of punctuations in your file. For example, a comma should be added in the sentence of "As stated in the introduction the step size for this algorithm is determined externally to the algorithm itself"
• Clear explanations are needed to present the meaning of all parameters/variables used in equations.
• The table would be better by being transposed.
• Please provide more references to support the statements in the Application Section.
• Please include the pseudocode for your algorithm.
• Raw data takes too much space in the current form. Please consider transposing it or find other ways to solve this issue.
• Wikipedia is not a proper citation source. Please avoid citing Wikipedia.

Dynamic optimization

• For the algorithm description, it would be better to have a pseudocode or a flow chart to summarize it.
• Attaching code to the Wiki editing page is not recommended.
• It would be better to provide more references for supporting the statements made in the Application Section.
• Please try to add more citations in the introduction section.
• In-text citations are required.
• Avoid using contraction (e.g., you've, don't) and pronouns (e.g., we, us) in scientific writing.
• Please add some mathematical expressions to Algorithm Discussion section for explicitness.
• Adding a different type of problem would be recommended instead of making Knapsack Problem larger size.
• In-text citation should be added to figure captions.

Nondifferentiable Optimization

• If the topic is about nondifferntial optimization, then it would be strange that only one approach is included in the Wiki page.
• It is also recommended to revise the citation format in the file based on our example files.
• Please add the case of non convex functions.
• Please add references to support the content in the Introduction section. In-text citations are required.
• Please use Latex equation editor for typing symbols and equations.
• There are multiple methods introduced in the Introduction section. Please expand the Algorithm Discussion section and add more details.
• The provided example is somewhat trivial. Please provide a more sophisticated example.
• It should be "CVXPY" instead of "CVXpy" and "MATLAB" instead of "Matlab" in Application and Conclusion sections.

Stackelberg leadership model

• Please make citations immediately after the citing contents. Please double-check all the citations in your file.
• Please include more citations in the application section to support your statements.
• Please use flowchart/pseudocode for representing the steps of algorithm
• Check the consistency of abbreviations (e.g. what is PAWS?)
• Sections 5-7 can be combined into one section and divided by subsections.

Quadratic constrained quadratic programming

• Please double-check that all cited Figures are clearly attached with citations.
• Please double-check the citations of your references.
• Need significant amount of citation to support the statement
• Mentioned the ways to relax the non convex set, so please provide at least one example either from SDP or RLT.
• Double check with abbreviations (e.g. KKT and SDP should be defined at the beginning sections)
• Equations should be written in a formal way (e.g. 1/2 should be 1 on top and 2 on the bottom)
• Abbreviations should be introduced only once throughout all sections (e.g., QCQP, QP, SDP)
• Avoid using pronouns (e.g., we) in scientific writing.
• Please revise “Ex. Objective” and “Ex. Constraint” parts in a more professional way in the Application section.

Derivative free optimization

• Please avoid the use of the term like "you" or "we" in the Wiki file.
• Please include more citations to support your statements in the application section.
• Please add more citations to support your statement in the introduction section
• Please try to include a flowchart or pseudocode for illustration of the algorithm.
• If DFO is defined in the previous sections, please use such abbreviations consistently (same for other abbreviations).
• Please add subheadings in the Application section for better readability.

Signomial problems

• The contents are good, but need to improve the clarity of the Wiki page.
• The citations need to be numbered by the order of their appearance.
• Provide more details on the Global Optimization Techniques and Metaheuristics approach that you mentioned.
• Please provide step-by-step calculation processes for the numerical examples.
• Please check some minor grammer issues in the file.
• For applications, please try to use coherent paragraphs instead of the lists of bullet items.
• 1, 2, and n in "x = [x1, x2, ..., xn]" should be subscripted in the Introduction section.
• "Minimize" and "Subject to" should not be italicized in equations.

Adadelta

• Adding a subsection title to divide the numerical examples will improve the clarity of this file.
• Please add the platforms (e.g. PyTorch, TF) to illustrate how this algorithm is introduced.
• Avoid using contraction (e.g., it’s, doesn’t) and pronouns (e.g., we, let's) in scientific writing.
• Citation of each algorithm should be added in the sentence "such as RMSprop, Adam, and Nadam, exist as well."
• Abbreviations should be introduced one time only throughout all sections (e.g., RNNs).

Adafactor

• The clarity of the alghrithm and numerical example session needs to be improved. It is convenient to list all equations, but not good to present it to others.
• Please revise your reference format.
• More citations are needed for supporting your statement in the introduction section.
• For the section "software tools and platforms" please provide more details, e.g. how PyTorch is using such optimizers and how this platform introduced this algorithm (do not directly copy and remember to add citation if this is not your own idea).
• The sentence "This article mainly introduces Adafactor and its function, algorithm, and application." can be removed.
• Algorithm and theory section is missing.

AdamW

• Please double-check if the citations are correctly formatted in the text.
• For the application section, it would be good to emphasize the advantages of AdamW compared to other approach by citing the quantitative results from previous literature.
• Please provide flowchart/pseudocode for representing your procedure
• Please provide citations for your statement (e.g. how AdamW used in Finance, where is your source)
• Avoid using pronouns (e.g., we, let's) in scientific writing.
• The code snippet in the Application section can be removed.

FTRL algorithm

• "The FTRL (Follow the Regularized Leader) family of algorithms are fundamental algorithms utilized in online learning and are a type of FTL (Follow the Leader) algorithm, which chooses a weight function at each timestep that minimizes the loss of all previously observed data, but implementations of the FTRL algorithm generally utilize a linearized loss function to reduce computational complexity, with a regularizer preventing the solution from diverging. " This sentence reads a little bit strange. Please revise it.
• It should be "Google" instead of "google".
• It should be "data point" instead of "datapoint".
• Import figure as equation is ok now. But for the real Wiki editing page, it should be done by equation form and should be properly worked with citations if any reference is used.
• Please try to improve the clarity and logic flow of the algorithm description part.
• "Other varieties of regularizers utilize both L1 and L2 loss, generating a stable and sparse solution. They are typically used in place of other gradient descent algorithms such as Objective Constraint Online Gradient Descent (OC-OGD) that induce sparsity into the model. An instance of the FTRL algorithm using both an L1 and L2 regularizers are shown below. " Please revise the grammer of the above texts.
• "The first application that the FTRL algorithm was used for was in(deleted) online advertising by Google"
• It is suggested to include pseudocode for this algorithm.
• Some equations are shown in figure form. Please use Latex equation editor for equations.

LossscaleOptimizer

• Provide more citations to support your statements in the Application section?
• Provide a figure to demonstrate the results generated from the numerical example?
• Please provide more applications
• Please provide the platform which used such algorithms
• The Conclusion section is missing.

Nadam

• Instead of providing python codes, it would be better to illustrate the calculation process and just provide the result Figures.
• If an abbr is not used in the following texts, it should not be defined. Such as "LSTM".
• Can you provide more explanations on how does NADAM improve the performance of the machine learning models that you mentioned?
• For introduction, no need to give so many words on introducing other methods.
• Please try to replace all your source code with a pseudocode and put it into algorithm section (that will be better for audiences to understand).
• Avoid using pronouns (e.g., we) in scientific writing.
• In Numerical Examples, t = t+1 = 1 is a coding syntax and not the proper math expression. Please use math expression here. This is the same for other time step updates.

Beyesian optimization

• Only suggestion is that providing code may not be appropriate in the Wiki page (all on your decision).
• Citation number should be double checked.
• You can use abbreviations if some special terms appeared multiple times (e.g. Bayesian Optimization -> BO).
• Avoid using pronouns (e.g., we) in scientific writing.
• There is no need to show your own code in this wiki page.
• Avoid adding citations in the conclusion section

Genetic algorithm

• Are the Figures used in the text self-generated or cited? If cited, please add citations. Also the resolution of the figures may need to be improved. Looks like two Figure 1 are included, please fix it.
• Please provide some citations for supporting your statement (e.g. in Introduction)
• Abbreviations should be consistent throughout the context (e.g. GA)
• Please avoid citing or adding links to Wikipedia.
• Please remove the sentence, "as documented in Computational Optimization and Applications."
• More details are expected for Algorithm Discussion section.

Simulated annealing

• Please provide citations in the application section to support your statements.
• Provide some figures for the numerical study
• Substitute those symbols in a formal way (e.g. T_min should be in a formal way).
• Once the abbr. is defined please use it throughout the context (e.g. SA)
• There are many extra words throughout the sections, (e.g., “Spaces” in the last sentence of Introduction section, “D.” in section 2.1)
• Avoid using pronouns (e.g., we) in scientific writing.

Particle swarm optimization

• "The concept was presented at the IEEE conference on neural networks in 1995" Is a citation needed here?
• "Relevance to modern optimization problems makes PSO an interesting research area."
• What does the square symbol mean in the x_best,p symbol. Same issues also are found in the following texts.
• "The convergence of the algorithm can be checked in a few different ways"
• It should be "hyperparameters" instead of "hyper parameters"
• It should be "Python" instead of "python"
• It may not be good to provide code in the Wiki editing page.
• In the equation, please try to use symbols only. One example could be in page 13, you can mention pbo in previous context not in the form of equations. Please check this throughout the context.
• "Depending on how the code is set up, scaling and offset factors may be required."
• Import a figure as function is acceptable now, but should be completed in all equation format when doing on the Wiki page.
• Please revise the reference formate based on our provided examples.
• Figure 1 can be revised by making the first letter of "converged?" capitalized and by centering the text for each box.
• Avoid using pronouns (e.g., we) in scientific writing.